WO2024012054A1 - 图像处理方法、智能终端及存储介质 - Google Patents

图像处理方法、智能终端及存储介质 Download PDF

Info

Publication number
WO2024012054A1
WO2024012054A1 PCT/CN2023/095627 CN2023095627W WO2024012054A1 WO 2024012054 A1 WO2024012054 A1 WO 2024012054A1 CN 2023095627 W CN2023095627 W CN 2023095627W WO 2024012054 A1 WO2024012054 A1 WO 2024012054A1
Authority
WO
WIPO (PCT)
Prior art keywords
template
partition
image
current
similar
Prior art date
Application number
PCT/CN2023/095627
Other languages
English (en)
French (fr)
Inventor
刘雨田
Original Assignee
深圳传音控股股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳传音控股股份有限公司 filed Critical 深圳传音控股股份有限公司
Publication of WO2024012054A1 publication Critical patent/WO2024012054A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability

Definitions

  • This application relates to the field of image processing technology, and specifically relates to an image processing method, an intelligent terminal and a storage medium.
  • prediction of image blocks is a very core technology.
  • the difference between the actual pixel value and the predicted pixel value can be obtained through prediction, and this value is encoded as the prediction residual for efficient transmission;
  • the decoding end can decode to obtain the prediction residual, and
  • the same prediction method is used at the encoding end to obtain the same prediction value, and then the image is reconstructed based on the prediction residual and prediction value.
  • this application provides an image processing method, an intelligent terminal and a storage medium, which can flexibly use the target prediction mode to ensure the accuracy of the prediction results.
  • this application provides an image processing method, which can be applied to smart terminals, including the following steps:
  • S2 Determine the prediction result of the image block according to the target prediction mode.
  • the image block is divided into a first partition and a second partition
  • the target prediction mode includes a first intra prediction mode, a first inter prediction mode, a second intra prediction mode, and a second inter prediction mode. At least one of the prediction modes.
  • the first partition adopts a first intra prediction mode or a first inter prediction mode
  • the second partition adopts a second intra prediction mode or a second inter prediction mode.
  • the first inter prediction mode is a unidirectional prediction mode or a bidirectional prediction mode
  • the second inter prediction mode is a unidirectional prediction mode or a bidirectional prediction mode.
  • the target prediction mode includes a template matching-based geometric division mode, and the image block is divided into a first partition and a second partition under the template matching-based geometric division mode.
  • the type of the geometric partitioning mode based on template matching includes: intra prediction based on template matching and/or inter prediction based on template matching; the first partition adopts the geometric partitioning mode based on template matching. Intra prediction based on template matching or inter prediction based on template matching; the second partition adopts the intra prediction based on template matching or the inter prediction based on template matching.
  • the target prediction mode includes a geometric division mode based on template matching.
  • the step S2 includes: determining that each partition in the image block corresponds to at least one similar template in the current template through a preset strategy; according to The similarity template determines the prediction result of the image block.
  • determining that each partition in the image block corresponds to at least one similar template in the current template through a preset strategy includes: determining the current template from a predetermined search range according to the type of the geometric division pattern based on template matching. Corresponding to at least one similar template.
  • the predetermined search range is determined based on a similar template corresponding to the reference block and/or the position of the prediction candidate block of the reference block.
  • the method further includes: determining the predetermined search range based on the positional relationship between the reference block and the image block and the position of the similar template corresponding to the reference block; and/or, The predetermined search range is determined according to the positional relationship between the reference block and the image block and the position of the prediction candidate block of the reference block.
  • determining at least one similar template corresponding to the current template from a predetermined search range according to the type of the geometric division pattern based on template matching includes: if the type of the geometric division pattern based on template matching is template-based Matching intra-frame prediction, then determine at least one similar template corresponding to the current template from the image where the image block is located; and/or, if the type of the geometric division mode based on template matching is inter-frame prediction based on template matching , then at least one similar template corresponding to the current template is determined from the reference image corresponding to the image where the image block is located.
  • determining at least one similar template corresponding to the current template from the image where the image block is located includes: determining from the reconstructed image area in the image where the image block is located according to preset search rules. A similar template that matches the current template; and at least one similar template corresponding to the current template is determined based on the matched similar template.
  • determining at least one similar template corresponding to the current template from the reference image corresponding to the image where the image block is located includes: determining the current template from the first reference image corresponding to the image where the image block is located. The first similar template corresponding to the template, and/or, determine the second similar template corresponding to the current template from the second reference image corresponding to the image where the image block is located, and combine the first similar template and/or the third similar template.
  • Two similar templates are determined as at least one similar template corresponding to the current template; and/or, at least one initial similar template is determined from the reference image corresponding to the image where the image block is located according to at least one initial motion vector, including any one Search for similar templates that match the current template within a predetermined range of the initial similar templates, and determine at least one similar template corresponding to the current template based on the matched similar templates.
  • Determining a similar template that matches the current template from the reconstructed image area in the image where the image block is located according to the preset search rules includes: searching for the reconstructed image area in the image where the image block is located. During the process, when a preset number of target areas that meet the matching conditions appear, each target area that meets the matching conditions is determined as a similar template that matches the current template; and/or, after the search for the reconstructed image area is completed, Determine a preset number of target areas that meet the matching conditions as similar templates that match the current template;
  • Determining at least one similar template corresponding to the current template according to the matched similar template includes: determining a template cost between the matched similar template and the current template, and determining the template cost corresponding to the preset template condition.
  • the matched similar template is determined to be at least one similar template corresponding to the current template.
  • determining the prediction result of the image block based on the similar template includes: determining the prediction result based on at least one similar template corresponding to the partition. at least one prediction candidate block of a partition; determining a prediction result of the image block based on the at least one prediction candidate block of each partition in the image block.
  • the method further includes: obtaining an initial current template of the image block; dividing the initial current template into a current template corresponding to the first partition of the image block and/or a current template of the image block.
  • step S1 includes at least one of the following: determining the target prediction mode of the image block through the first flag; determining the prediction mode corresponding to the minimum rate distortion cost as the target prediction mode of the image block.
  • a first flag is used to indicate that the image block uses the geometric division mode based on template matching, and using at least one second flag to indicate the prediction processing used by each of the first partition and/or the second partition of the image block.
  • the S2 step includes: determining a weighted prediction candidate block corresponding to each partition in at least one partition of the image block according to the target prediction mode, so as to determine the image block. Block prediction results.
  • step S2 includes: determining an initial current template of the image block for determining the initial current template. Prediction results of image blocks; optionally, the initial current template includes a partitioned template area and a non-partitioned template area.
  • the partition template area includes a first current template corresponding to the first partition and a second current template corresponding to the second partition.
  • the initial current template includes at least one of an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left side of the image block. ;
  • the initial current template includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the abscissa range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition; or, the ordinate range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition.
  • the ordinate range of the current template corresponding to the second partition is different; or, the length of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition is different from that of the image block and/or the length of the current template corresponding to the second partition.
  • the lengths of the prediction candidate blocks of the image block are not equal; or, the width of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition are different from those of the image block and/or the image.
  • the prediction candidate blocks of the blocks are not equal in width.
  • the current template corresponding to the first partition of the image block and/or the current template corresponding to the second partition of the image block includes at least one boundary, and the at least one boundary is located on the dividing line used by the image block. on the straight line; or,
  • the initial current template includes a current template corresponding to the first partition of the image block, a current template corresponding to the second partition of the image block, and a non-partition template area, where the straight line where the dividing line used by the image block is located passing through the non-partitioned template area; or the non-partitioned template area is adjacent to at least one of the current template corresponding to the first partition of the image block and the current template corresponding to the second partition of the image block.
  • the type of the current template includes a beveled edge template and/or a non-beveled edge template, and the type of the at least one similar template is the same as the type of the current template; when the type of the current template is the beveled edge template, When the edge template is a side template, the hypotenuse position and slope of the hypotenuse template are set according to the dividing line dividing the image block; when the type of the current template is the non-hypotenuse template, the position of the non-hypotenuse template Set according to the area where the straight line of the dividing line passes through the initial current template.
  • This application also provides a device including:
  • An acquisition module used to acquire or determine the target prediction mode of the image block
  • a determining module configured to determine a prediction result of the image block according to the target prediction mode.
  • the present application also provides an intelligent terminal, including: a memory and a processor, wherein an image processing program is stored on the memory, and when the image processing program is executed by the processor, any one of the above image processing methods is implemented. step.
  • the present application also provides a computer-readable storage medium, which stores a computer program.
  • the computer program is executed by a processor, the steps of any of the above image processing methods are implemented.
  • the image processing method of this application can be applied to smart terminals, including the steps: S1: obtain or determine the target prediction mode of the image block; S2: determine the prediction result of the image block according to the target prediction mode.
  • the target prediction mode used by the image block is any prediction mode that meets the conditions.
  • the prediction mode can be determined from at least one prediction mode, and the selection flexibility is high; the target prediction mode can correspond to one or more prediction processes.
  • image blocks can be flexibly predicted through the target prediction mode to ensure the accuracy of the prediction results of image blocks. It can be seen that using this solution can realize the function of flexibly using the target prediction mode, ensuring the accuracy of the prediction results, and solving the problems of inflexible prediction modes and limited prediction processing methods.
  • Figure 1 is a schematic diagram of the hardware structure of a mobile terminal that implements various embodiments of the present application
  • FIG. 2 is a communication network system architecture diagram provided by an embodiment of the present application.
  • Figure 3a is a schematic flowchart of an image processing method according to the first embodiment
  • Figure 3b is a schematic diagram showing the effect of dividing image blocks by a dividing line according to the first embodiment
  • Figure 4a is a schematic flowchart of an image processing method according to a second embodiment
  • Figure 4b is a schematic diagram of angle division according to the second embodiment
  • Figure 4c is a schematic diagram showing various offsets corresponding to angle ⁇ i according to the second embodiment
  • Figure 4d is a schematic diagram of a division mode according to the second embodiment
  • Figure 5a is a schematic diagram of an initial current template according to the second embodiment
  • Figure 5b is a schematic diagram showing the effect of dividing the initial current template according to the second embodiment
  • Figure 5c is a schematic diagram showing another effect of dividing the initial current template according to the second embodiment
  • Figure 6 is a schematic flowchart of an image processing method according to a third embodiment
  • Figure 7a is a reconstructed image area in the image where the image block is located according to the third embodiment
  • Figure 7b is a schematic diagram of intra prediction based on template matching according to the third embodiment.
  • Figure 7c is a schematic diagram of inter-frame prediction based on template matching according to the third embodiment.
  • Figure 7d is a schematic diagram of adjacent block positions of a motion vector candidate list according to the third embodiment.
  • Figure 7e is a schematic diagram of a motion vector candidate list according to the third embodiment.
  • Figure 7f is a schematic diagram of another inter-frame prediction based on template matching according to the third embodiment.
  • Figure 7g is a schematic diagram illustrating inter-frame prediction based on template matching for each partition according to the third embodiment
  • Figure 7h is a prediction schematic diagram when different types of geometric division modes based on template matching are used according to the third embodiment.
  • Figure 8 is a schematic flowchart of an image processing method according to the fourth embodiment.
  • Figure 9 is a schematic structural diagram of an image processing device according to an embodiment of the present application.
  • Figure 10 is a schematic diagram of the hardware structure of a controller according to an embodiment of the present application.
  • Figure 11 is a schematic diagram of the hardware structure of a network node according to an embodiment of the present application.
  • first, second, third, etc. may be used herein to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • first information may also be called second information, and similarly, the second information may also be called first information.
  • word “if” as used herein may be interpreted as “when” or “when” or “in response to determining.”
  • singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context indicates otherwise.
  • A, B, C means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C"; another example is, “ A, B or C” or "A, B and/or C” means "any of the following: A; B; C; A and B; A and C; B and C; A and B and C". Exceptions to this definition occur only when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some manner.
  • each step in the flow chart in the embodiment of the present application is displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least some of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and their execution order is not necessarily sequential. may be performed in turn or alternately with other steps or sub-steps of other steps or at least part of stages.
  • the words “if” or “if” as used herein may be interpreted as “when” or “when” or “in response to determination” or “in response to detection.”
  • the phrase “if determined” or “if (stated condition or event) is detected” may be interpreted as “when determined” or “in response to determining” or “when (stated condition or event) is detected )” or “in response to detecting (a stated condition or event)”.
  • step codes such as S301 and S302 are used for the purpose of describing the corresponding content more clearly and concisely, and do not constitute a substantial restriction on the sequence. Those skilled in the art may S302 will be executed first and then S301, etc., but these should be within the protection scope of this application.
  • Smart terminals can be implemented in various forms.
  • the smart terminals described in this application may include mobile phones, tablet computers, notebook computers, PDAs, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, Smart terminals such as wearable devices, smart bracelets, and pedometers, as well as fixed terminals such as digital TVs and desktop computers.
  • PDA Personal Digital Assistant
  • PMP portable media players
  • navigation devices Smart terminals such as wearable devices, smart bracelets, and pedometers
  • Smart terminals such as wearable devices, smart bracelets, and pedometers
  • fixed terminals such as digital TVs and desktop computers.
  • a mobile terminal will be taken as an example.
  • the structure according to the embodiments of the present application can also be applied to fixed-type terminals.
  • the mobile terminal 100 may include: an RF (Radio Frequency, radio frequency) unit 101, a WiFi module 102, an audio output unit 103, and a /V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components.
  • RF Radio Frequency, radio frequency
  • the radio frequency unit 101 can be used to receive and send information or signals during a call. Specifically, after receiving the downlink information of the base station, it is processed by the processor 110; in addition, the uplink data is sent to the base station.
  • the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, transceiver, coupler, low noise amplifier, duplexer, etc.
  • the radio frequency unit 101 can also communicate with the network and other devices through wireless communication.
  • the above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service, General Packet Wireless Service), CDMA2000 (Code Division Multiple Access 2000, Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access, Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple) Access, Time Division Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, Frequency Division Duplex Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution, Time Division Duplex Long Term Evolution) and 5G etc.
  • GSM Global System of Mobile communication
  • GPRS General Packet Radio Service, General Packet Wireless Service
  • CDMA2000 Code Division Multiple Access 2000, Code Division Multiple Access 2000
  • WCDMA Wideband Code Division Multiple Access
  • TD-SCDMA Time Division-Synchronous Code Division Multiple
  • WiFi is a short-distance wireless transmission technology.
  • the mobile terminal can help users send and receive emails, browse web pages, access streaming media, etc. through the WiFi module 102. It provides users with wireless broadband Internet access.
  • FIG. 1 shows the WiFi module 102, it can be understood that it is not a necessary component of the mobile terminal and can be omitted as needed without changing the essence of the invention.
  • the audio output unit 103 may, when the mobile terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, etc., receive the audio signal received by the radio frequency unit 101 or the WiFi module 102 or store it in the memory 109 The audio data is converted into audio signals and output as sound. Furthermore, the audio output unit 103 may also provide audio output related to a specific function performed by the mobile terminal 100 (eg, call signal reception sound, message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.
  • the A/V input unit 104 is used to receive audio or video signals.
  • the A/V input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042.
  • the graphics processor 1041 can process still pictures or images obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. Video image data is processed.
  • the processed image frames may be displayed on the display unit 106.
  • the image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage media) or sent via the radio frequency unit 101 or WiFi module 102.
  • the microphone 1042 can receive sounds (audio data) via the microphone 1042 in operating modes such as a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sounds into audio data.
  • the processed audio (voice) data can be converted into a format that can be sent to a mobile communication base station via the radio frequency unit 101 for output in a phone call mode.
  • Microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and transmitting audio signals.
  • the mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors.
  • the light sensor includes an ambient light sensor and a proximity sensor.
  • the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light.
  • the proximity sensor can turn off the display when the mobile terminal 100 moves to the ear. Panel 1061 and/or backlight.
  • the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three axes). It can detect the magnitude and direction of gravity when stationary.
  • It can be used to identify applications of mobile phone posture (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the mobile phone, it can also be configured with fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, Other sensors such as thermometers and infrared sensors will not be described in detail here.
  • the display unit 106 is used to display information input by the user or information provided to the user.
  • the display unit 106 may include a display panel 1061, which may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
  • LCD Liquid Crystal Display
  • OLED Organic Light-Emitting Diode
  • the user input unit 107 may be used to receive input numeric or character information, and generate key signal input related to user settings and function control of the mobile terminal.
  • the user input unit 107 may include a touch panel 1071 and other input devices 1072.
  • the touch panel 1071 also known as a touch screen, can collect the user's touch operations on or near the touch panel 1071 (for example, the user uses a finger, stylus, or any suitable object or accessory on or near the touch panel 1071 operation), and drive the corresponding connection device according to the preset program.
  • the touch panel 1071 may include two parts: a touch detection device and a touch controller.
  • the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into contact point coordinates , and then sent to the processor 110, and can receive the commands sent by the processor 110 and execute them.
  • the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave.
  • the user input unit 107 may also include other input devices 1072.
  • other input devices 1072 may include but are not limited to one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, etc., which are not specifically discussed here. limited.
  • the touch panel 1071 can cover the display panel 1061.
  • the touch panel 1071 detects a touch operation on or near it, it is transmitted to the processor 110 to determine the type of the touch event, and then the processor 110 determines the type of the touch event according to the touch event.
  • the type provides corresponding visual output on the display panel 1061.
  • the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated. The implementation of the input and output functions of the mobile terminal is not limited here.
  • the interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100 .
  • external devices may include a wired or wireless headphone port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, audio input/output (I/O) port, video I/O port, headphone port, etc.
  • the interface unit 108 may be used to receive input (eg, data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to connect between the mobile terminal 100 and an external device. Transfer data between devices.
  • Memory 109 may be used to store software programs as well as various data.
  • the memory 109 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.;
  • the storage data area may Store data created based on the use of the mobile phone (such as audio data, phone book, etc.), etc.
  • memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the processor 110 is the control center of the mobile terminal, using various interfaces and lines to connect various parts of the entire mobile terminal, by running or executing software programs and/or modules stored in the memory 109, and calling data stored in the memory 109 , execute various functions of the mobile terminal and process data, thereby overall monitoring the mobile terminal.
  • the processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor.
  • the application processor mainly processes the operating system, user interface, application programs, etc., and modulation
  • the demodulation processor mainly handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 110 .
  • the mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components.
  • a power supply 111 such as a battery
  • the power supply 111 may be logically connected to the processor 110 through a power management system, thereby achieving management of charging, discharging, and power consumption management through the power management system. and other functions.
  • the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described again here.
  • FIG. 2 is an architecture diagram of a communication network system provided by an embodiment of the present application.
  • the communication network system is an LTE system of universal mobile communication technology.
  • the LTE system includes UEs (User Equipment, User Equipment) connected in sequence. )201, E-UTRAN (Evolved UMTS Terrestrial Radio Access Network, Evolved UMTS Terrestrial Radio Access Network) 202, EPC (Evolved Packet Core, Evolved Packet Core Network) 203 and the operator's IP business 204.
  • UEs User Equipment, User Equipment
  • E-UTRAN Evolved UMTS Terrestrial Radio Access Network
  • EPC Evolved Packet Core, Evolved Packet Core Network
  • UE201 may be the above-mentioned mobile terminal 100, which will not be described again here.
  • E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc.
  • eNodeB2021 can be connected to other eNodeB2022 through backhaul (for example, X2 interface), eNodeB2021 is connected to EPC203, and eNodeB2021 can provide access from UE201 to EPC203.
  • backhaul for example, X2 interface
  • EPC 203 may include MME (Mobility Management Entity, mobility management entity) 2031, HSS (Home Subscriber Server, home user server) 2032, other MME 2033, SGW (Serving Gate Way, service gateway) 2034, PGW (PDN Gate Way, packet data Network Gateway) 2035 and PCRF (Policy and Charging Rules Function, policy and charging functional entity) 2036, etc.
  • MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management.
  • HSS2032 is used to provide some registers to manage functions such as the home location register (not shown in the figure), and to save some user-specific information about service characteristics, data rates, etc. All user data can be sent through SGW2034.
  • PGW2035 can provide IP address allocation and other functions for UE201.
  • PCRF2036 is the policy and charging control policy decision point for business data flows and IP bearer resources. It is a policy and charging execution functional unit (Not shown) Select and provide available policy and billing control decisions.
  • IP services 204 may include the Internet, Intranet, IMS (IP Multimedia Subsystem, IP Multimedia Subsystem) or other IP services.
  • IMS IP Multimedia Subsystem, IP Multimedia Subsystem
  • Figure 3a is a schematic flowchart of an image processing method according to the first embodiment.
  • the execution subject in this embodiment can be a computer device or a cluster composed of multiple computer devices.
  • the computer device can It may be an intelligent terminal (such as the aforementioned mobile terminal 100), or it may be a server.
  • the execution subject in this embodiment is an intelligent terminal as an example for explanation.
  • the image block in step S301 refers to the image block currently being encoded in the input video image (ie, the video frame), which may also be called the current block, or the current image block, or the current encoding block.
  • the image block can be a Coding Tree Unit (CTU) in the input video image, or a Coding Unit (CU), or a Transform Unit (Transform Unit, TU), etc. There are no restrictions on this.
  • the image blocks can be square blocks (that is, the size of the image blocks is square) or non-square blocks. Non-square blocks may be rectangular-sized image blocks, including horizontal blocks (width greater than height) and/or vertical blocks (height greater than width). For example, when the image block is a CU, the CU may be a square block or a non-square block. . There are no restrictions here. In this application, unless otherwise specified, the image block and the current block are equivalent.
  • a prediction mode can usually be used to indicate a specific prediction processing method.
  • the prediction processing of image blocks generally includes intra-frame prediction and inter-frame prediction.
  • other new prediction modes can also be formed through various deformation combinations to cope with different video encoding and decoding scenarios.
  • the geometric partition mode (GPM mode) is used for the boundary parts of moving objects in the image. This geometric partition mode divides image blocks into image blocks by using dividing lines that fit the boundaries of the objects. Divide into different partitions for inter-frame prediction.
  • the target prediction mode may be any prediction mode among at least one prediction mode supported by the image block. This prediction mode is used to predict image blocks to obtain prediction results.
  • the target prediction mode includes at least one of an intra prediction mode or an inter prediction mode.
  • the intra prediction mode or inter prediction mode is a unidirectional prediction mode or a bidirectional prediction mode. Unidirectional prediction mode corresponds to one motion vector, and bidirectional prediction mode corresponds to two motion vectors.
  • the image block is divided into a first partition and a second partition. The first partition adopts a first intra prediction mode or a first inter prediction mode, or the second partition adopts a second intra prediction mode or a first inter prediction mode. Two inter prediction modes.
  • the target prediction mode includes at least one of a first intra prediction mode, a first inter prediction mode, a second intra prediction mode, and a second inter prediction mode.
  • the target prediction mode includes a geometric partitioning mode based on template matching.
  • the geometric division mode based on template matching is a new type of geometric division mode, which can also be called the GPM mode with template matching (Template Matching, TM). It not only has the function of geometric division mode, that is, dividing image blocks into different image areas through dividing lines, but can also be combined with templates for prediction.
  • Template matching refers to predicting image blocks by using prediction candidate blocks determined by using a template.
  • the template refers to the area associated with the image block, such as the coded area where the pixels adjacent to the left boundary of the image block are located.
  • the geometric division mode based on template matching applies template matching to the GPM mode, and can perform corresponding prediction processing on different image areas of the image block in the GPM mode, thereby obtaining the prediction result of the entire image block.
  • the intra prediction mode may be intra prediction based on template matching
  • the inter prediction mode may be inter prediction based on template matching.
  • the image block is divided into a first partition and a second partition in the geometric partition mode based on template matching.
  • the geometric division mode based on template matching is essentially a geometric division mode, according to the principle of the geometric division mode, when the prediction mode used by the image block is the geometric division mode based on template matching, it can be divided by the dividing line in the GPM mode Image blocks to obtain different image areas, where the image area includes a first partition and a second partition.
  • the first partition and the second partition are rectangular or non-rectangular areas in the image block, and the first partition and the second partition are relative.
  • the first partition and the second partition may be rectangular, triangular, or trapezoidal areas with respect to the image block.
  • a horizontal dividing line is used to divide the image block.
  • the image area above the horizontal dividing line can be called the first partition, and the image area below the horizontal dividing line can be called the first partition.
  • the image area may be called the second partition.
  • the image area below the horizontal dividing line may also be called a first partition, and the image area above the horizontal dividing line may be called a second partition, and both partitions are rectangular areas.
  • two partitions obtained by dividing the image block by the dividing line one is a triangular area and the other is a non-rectangular area.
  • the geometric division mode based on template matching may include at least one type, and each type may indicate logic for prediction processing using rules of template matching in the geometric division mode.
  • each partition in at least one partition of the image block (such as the first partition and/or the second partition)
  • each partition can adopt a geometric division mode based on template matching, but the specific type can be freely selected, and the prediction flexibility is high.
  • the first partition and the second partition adopt the same or different types of geometric partition modes based on template matching.
  • each partition in the image block adopts the same type of geometric division mode based on template matching, then each partition uses the same prediction processing logic to determine the prediction result of the image block; if each partition in the image block adopts a geometric division mode based on template matching. If the type is different, each partition uses different prediction processing logic. Determine the prediction result of the image block. It can be seen that at least one type is provided for the geometric partitioning mode based on template matching, and the types that can be used by different partitions support flexible combination, thereby increasing the flexibility of GPM mode prediction with TM.
  • the type of geometric division mode based on template matching includes: intra prediction based on template matching and/or inter prediction based on template matching.
  • Intra-frame prediction based on template matching is a method of prediction that uses the correlation of image blocks within a frame
  • inter-frame prediction based on template matching is a method of prediction that uses the correlation of images between frames.
  • Intra-frame prediction based on template matching can search for a template matching the template of the currently encoded image block in the encoded image where the currently encoded image block is located in the geometric partition mode for prediction;
  • inter-frame prediction based on template matching can In the geometric partition mode, the reference frame corresponding to the currently encoded image block is searched for a template that matches the template of the currently encoded image block for prediction.
  • the template of the above image block can be the template corresponding to any partition in the image block.
  • Inter prediction based on template matching is specifically a geometric division mode of inter prediction based on template matching
  • intra prediction based on template matching is specifically a geometric division mode of intra prediction based on template matching.
  • any of the above types of geometric partition modes based on template matching can be used, that is, the first partition adopts the geometric partition mode based on template matching.
  • the first and second partitions adopt the same type of geometric partition mode based on template matching, including: 1 Both the first and second partitions adopt intra prediction based on template matching, 2 Both the first and second partitions adopt Inter prediction based on template matching.
  • the first partition and the second partition adopt different types of geometric partition modes based on template matching, including: 1 The first partition adopts inter prediction based on template matching, and the second partition adopts intra prediction based on template matching; 2 The first partition adopts template matching-based inter prediction.
  • One partition uses template matching-based intra prediction, and the second partition uses template matching-based inter prediction.
  • the GPM mode based on template matching is specifically a GPM mode that combines intra-frame prediction and inter-frame prediction based on template matching.
  • the GPM mode based on template matching is specifically a GPM mode based on intra-frame prediction based on template matching, or a GPM mode based on inter-frame prediction based on template matching. It can be seen that different partitions support at least one type of geometric partitioning mode based on template matching, and the same image block can be predicted using at least one prediction processing method (including inter-frame prediction and/or intra-frame prediction) in the same prediction mode, so that , the use of geometric division patterns based on template matching by image blocks can be more flexible and diverse, thereby increasing the flexibility of GPM pattern prediction with template matching.
  • the target prediction mode for an image block can be determined or obtained in a variety of ways, for example, based on the rate distortion cost of each prediction mode applied to the image block, or based on the prediction mode used by image blocks adjacent to the current block. OK wait.
  • the method of obtaining or determining the target prediction mode please refer to the following embodiments, which will not be described in detail here.
  • S302 Determine the prediction result of the image block according to the target prediction mode.
  • the target prediction mode may indicate a method for performing prediction processing on the image block.
  • the target prediction mode is a geometric division mode based on template matching
  • the prediction processing for the image block includes segmentation processing of the image block and template matching processing.
  • Prediction results can be obtained by predicting image blocks based on the target prediction mode.
  • the target prediction mode is a geometric division mode based on template matching, specifically a geometric division mode of inter-frame prediction based on template matching.
  • the reference frame corresponding to the image where the image block is located can be searched for the current frame.
  • a template matched by a template of an encoded image block is used to determine a prediction candidate block based on the matched template, and a prediction result is determined based on the prediction candidate block.
  • the prediction result of the image block may be determined according to the target prediction mode and the preset strategy.
  • the preset strategy is used to indicate the processing strategy for image blocks in the target prediction mode, and the preset strategy can match the target prediction mode.
  • the target prediction mode is a geometric division mode based on template matching
  • the preset strategy may be a strategy indicating template matching, specifically used to find a template that matches the template of the image block.
  • the image processing method provided by the embodiment of the present application can determine or obtain the target prediction mode of the image block.
  • the target prediction mode can be selected from at least one prediction mode, and the selection range is wide and the flexibility is high. Since the target prediction mode can indicate at least one way to perform prediction processing on the image block, the target prediction mode can flexibly use the corresponding prediction processing method to predict the currently encoded image block to ensure the accuracy of the prediction result.
  • the target prediction mode includes a geometric division mode based on template matching
  • the type of geometric division mode based on template matching not only includes inter prediction based on template matching, but also includes intra prediction based on template matching, and Different partitions in the image block can use the same or different types of geometric division modes based on template matching.
  • the image blocks can combine at least one type of geometric division mode based on template matching and use different prediction processing methods for prediction, so that the image block It is more flexible to use the geometric partitioning mode based on template matching.
  • Figure 4a is a schematic flowchart of an image processing method according to a second embodiment.
  • the execution subject in this embodiment can be a computer device or a cluster composed of multiple computer devices.
  • the computer device can It may be an intelligent terminal (such as the aforementioned mobile terminal 100), or it may be a server.
  • the execution subject in this embodiment is an intelligent terminal as an example for explanation.
  • S401 includes: determining a target prediction mode of the image block through a first flag.
  • the first flag may be a flag transmitted in the bit stream, or may be a flag transmitted in the CTU or CU.
  • the target prediction mode includes a geometric division mode based on template matching, and the first flag is used to indicate whether the image block uses a geometric division mode based on template matching (or in other words, whether the image block will be The template is applied to the geometric partitioning mode).
  • the first flag is 1, indicating that the image block uses the geometric division mode based on template matching, and/or, the second flag is 0, indicating that the image block does not use the geometric division mode based on template matching, but uses other predictions. model.
  • This method can be applied to the decoding end or the encoding end.
  • the first flag indicates that the image block uses a partitioning mode based on target matching
  • at least one second flag is used to indicate that the image block uses a target matching-based partitioning mode.
  • the second flag may be a coding unit level flag, and the second flag is used to indicate that each partition of the image block uses at least one of template matching-based intra prediction and template matching-based inter prediction.
  • the second flag indicates that the first partition in the image block uses template matching-based intra prediction, and the second partition uses template matching-based inter prediction.
  • One or more second flags can be used to indicate the above content.
  • the second flag is expressed in binary, specifically using a combination of two bits "1" and "0", such as 00, 01, and One bit indicates that one partition of the image block (for example, the first partition) uses template matching-based intra prediction or template matching-based inter prediction, and the second bit indicates that another partition of the image block (for example, the second partition) uses template-based prediction.
  • Matched intra prediction or template matching based inter prediction, 1 means using template matching based intra prediction, and/or, 0 means using template matching based inter prediction.
  • the second flag is 00, indicating that both the first and second partitions use template matching-based inter-frame prediction.
  • a second flag can be used to indicate the type of template matching-based geometric division mode used by each partition in the image block.
  • the second flag may also be represented by other characters, such as a combination of numbers and letters, with the number 1 representing the type of template matching-based geometric partitioning pattern of the first partition, and/or, 0 representing the second partition.
  • the type of geometric division mode based on template matching the letter a indicates the use of inter-frame prediction based on template matching, and the letter b indicates the use of intra-frame prediction based on template matching.
  • 1-a can be used to indicate that the first partition adopts template matching-based inter-frame prediction.
  • a plurality of second flags can be used to indicate the type of template matching-based geometric division mode adopted by each partition in the image block. It can be understood that the representation of the second mark is not limited to the manner introduced above. It can be seen that through the combined indication of the first flag and the second flag, the prediction mode used by the image block can be quickly and conveniently determined, which is very simple and convenient.
  • a first flag is used to indicate that the image block uses the geometric division based on template matching. mode, and using at least one second flag to indicate the prediction processing used by each of the first partition and/or the second partition of the image block.
  • the prediction processing used by the first partition and/or the second partition includes: intra prediction based on template matching and inter prediction based on template matching. At least a second flag indicates whether the first partition and/or the second partition of the image block uses at least one of template matching-based intra prediction and template matching-based inter prediction.
  • the first mark and the second mark here can be the same as those introduced previously, and will not be described again.
  • the GPM mode based on template matching and the prediction processing of the corresponding partition in this prediction mode can be directly and quickly determined through the indications of various flags. It is a very efficient method when applied to the encoding end or decoding end, which can quickly determine the prediction results and improve encoding or decoding efficiency.
  • S401 includes: determining the prediction mode corresponding to the minimum rate distortion cost as the target prediction mode of the image block. This method can be applied to the encoding side. First, the rate distortion cost of each prediction mode applied to the image block in at least one prediction mode can be determined, and then the minimum rate distortion cost corresponding to the minimum rate distortion cost can be determined by comparing the rate distortion costs of various prediction modes. prediction mode and use it as the target prediction mode of the image block.
  • the encoder may determine the color component of the current block (including the luminance component and/or the chrominance component); based on the parameters of the current block, use at least one prediction mode (such as intra prediction and/or inter prediction) to separately predict the color component. Predictive coding is performed, and the rate distortion cost corresponding to each prediction mode is calculated to determine the minimum rate distortion cost from the rate distortion cost corresponding to at least one prediction mode. Finally, the prediction mode corresponding to the minimum rate distortion cost is determined as the target prediction mode of the current block. When the prediction mode corresponding to the minimum rate distortion cost is the GPM mode based on template matching, the GPM mode based on template matching is determined as the target prediction mode of the current block.
  • the prediction mode corresponding to the minimum rate distortion cost is the GPM mode based on template matching
  • the GPM mode based on template matching is determined as the target prediction mode of the current block.
  • the template corresponding to each partition of the current block under the dividing line type of each GMP mode can be determined.
  • the rate distortion cost corresponding to the prediction result of the current block can be determined for use with other prediction modes.
  • the rate distortion cost corresponding to the prediction result of the current block is compared to determine the type of geometric division mode based on template matching in each partition of the current block.
  • the GPM mode based on template matching is a prediction mode that applies a template to the GPM mode
  • the dividing line type of the GPM mode can be determined first. Specifically, it includes the following content: By traversing 64 division models corresponding to the GPM mode, determine the division mode with the smallest rate distortion cost. Afterwards, the division mode with the smallest output rate distortion cost is used as the target division mode of the current block. For example, 64 partition models are shown in Table 1. Table 1 is a mapping table of GPM partition index gpm_partition_idx, angle index angleIdx and distance index distanceIdx.
  • determining the partition mode with the smallest rate distortion cost includes determining the target GPM partition index gpm_partition_idxT, the target angle index angleIdxT, and the target distance index distanceIdxT corresponding to the partition mode (ie, the target partition mode of the current block).
  • angle index angleIdx when the angle index angleIdx takes different values, it corresponds to different angles ⁇ i as shown in Figure 4b, where i ranges from 1 to 24.
  • distance index distanceIdx takes different values, it corresponds to ⁇ j in Figure 4c, where j ranges from 0 to 3.
  • the combination of different values of the angle index angleIdx and the distance index distanceIdx in Table 1 respectively constitute different division methods. The details can be shown in Figure 4d.
  • a lookup table can be used to represent the relationship between the GPM partition index gpm_partition_idx, the angle index angleIdx, and the distance index distanceIdx. For example, see Table 2 below.
  • the corresponding data can be quickly found based on a certain recorded index (for example, the angle index), thereby efficiently determining the dividing line type of the GPM mode.
  • a certain recorded index for example, the angle index
  • angle index angleIdx corresponds to the sine of the angle and the distance index distanceIdx corresponds to ⁇ j .
  • a mapping table about angleIdx and cos( ⁇ ) can be set, as shown in Table 3.
  • a mapping table between the intermediate variable of angleIdx and cos( ⁇ ) may also be set.
  • the encoder may encapsulate the prediction mode parameters corresponding to the target prediction mode into a bit stream for transmission.
  • the target prediction mode is a template matching-based GPM mode
  • the template matching-based GPM parameters corresponding to the template matching-based GPM mode can be binarized and then packaged into a bit stream for transmission.
  • the GPM parameter based on template matching here is a prediction mode parameter.
  • the prediction mode parameters include at least one of the following: information indicating whether to use a geometric division mode based on template matching (ie, prediction mode information), Prediction mode type information indicating a type of geometric division mode based on template matching and division mode type information indicating a division line type of the division mode.
  • the prediction mode parameter is a GPM parameter based on template matching. Through the information included in the prediction mode parameter, the GPM parameter based on template matching can be used to indicate whether the image block adopts the GPM mode based on template matching, the dividing line type of the GPM mode, and the template-based GPM parameter. Matches at least one of the GPM pattern types.
  • the prediction mode parameters can be obtained by parsing the received bit stream.
  • the prediction mode parameter includes a parameter indicating whether the current block (ie, the currently encoded image block) adopts a GPM mode based on template matching, specifically a parameter indicating that the current block adopts a GPM mode based on template matching, then determine GPM parameters of the current block.
  • the GPM parameter of the current block includes information indicating the dividing line type of the GPM mode.
  • the partition line type of the GPM mode may be represented by the GPM partition index gpm_partition_idx regarding the GPM parameter.
  • the corresponding angle index angleIdx and distance index distanceIdx can be determined through the GPM partition index gpm_partition_idx.
  • GPM parameters may also include other parameters.
  • the GPM parameter may also include a tag or index of the prediction type used by the first partition (i.e., the type of GPM pattern based on template matching corresponding to the first partition), and a tag or index of the prediction type used by the second partition (i.e., The type of GPM pattern based on template matching corresponding to the second partition).
  • the GPM parameter may also include a motion vector index in GPM mode.
  • gpm_idx0[x0][y0] represents the position of the first motion vector in the merge candidate list
  • gpm_idx1[x0][y0] represents the position of the second motion vector in the merge candidate list.
  • the first motion vector and the second motion vector may be motion vectors of the first partition, or the first motion vector and the second motion vector may be motion vectors of the second partition.
  • the tag or index of the prediction type used by the first partition indicates that the first partition adopts the inter prediction mode based on template matching
  • the tag or index of the prediction type used by the second partition indicates that the second partition adopts the inter prediction mode based on template matching.
  • the first motion vector and the second motion vector are motion vectors of the first partition.
  • the prediction method of the GPM mode based on template matching at the decoding end may include the following steps: (1) Obtain or determine prediction mode information from the bit stream, and the prediction mode information is used to indicate whether the image block adopts the GPM mode based on template matching. (2) If it is determined to use the GPM mode based on template matching, further determine the dividing line type of the GPM mode, the template of the current block, and the GPM prediction mode type based on template matching.
  • determining the GPM mode type based on template matching includes determining whether to use an intra prediction mode based on template matching, and/or determining whether to use an inter prediction mode based on template matching, which can be obtained from the bitstream or Determine template matching-based GPM prediction mode type information.
  • the template matching-based GPM prediction mode type information is used to indicate whether the image block uses the template matching-based intra prediction mode, and/or whether the template matching-based inter prediction mode is used. ; Determine the template of the image block including OK The size, shape, and partitions of each template. The determination of the template is related to the type of dividing line. For example, the position, shape, and corresponding partitions of the template can be determined based on the position of the dividing line. Afterwards, the prediction result of the image block can be determined according to the content introduced in S402 and S403.
  • the template Since the dividing line type of the GPM mode contains at least one type, the template also has at least one design style.
  • the specific prediction processing also includes intra-frame prediction and inter-frame prediction. By combining these contents, a GPM mode based on template matching can be constructed.
  • the matching GPM mode can be divided into many more detailed types. For example, 128 types can be combined based on 64 dividing line types and 2 prediction modes.
  • the GPM mode based on template matching greatly expands the types of prediction modes, thereby making the prediction modes used by each partition in the image block more selective, which helps to increase the flexibility of the geometric division mode based on template matching.
  • the target prediction mode includes a template matching based geometric partitioning mode.
  • the implementation of determining the prediction result of the image block according to the target prediction mode may include the content described in S402 and S403 below.
  • the implementation of determining the prediction result of the image block according to the target prediction mode may also be: determining the initial current template of the image block, To determine the prediction result of the image block through the initial current template.
  • the solutions described in S402 to S403 can also be used.
  • the image block can be divided based on the dividing line corresponding to the dividing line type included in the geometric dividing mode of the template matching to obtain multiple image areas, and these can be used as images individual partitions of the block.
  • Each partition in the image block has a corresponding template, which is called the current template here.
  • the current template includes the area where the coded pixels adjacent to the corresponding partition of the image block are located.
  • the shape of the current template can be a rectangle or another shape, such as a trapezoid.
  • the preset strategy here can be used to indicate criteria for matching other templates with the current template of the image block, and based on the criteria, similar templates that match the current template of the image block can be found.
  • the current template corresponding to each partition in the image block may be determined.
  • Each partition of the image block includes a first partition and a second partition.
  • the current template corresponding to each partition may be determined by: obtaining the initial current template of the image block; and determining the initial current template based on the dividing line. Divide to obtain the current template corresponding to the first partition of the image block and the current template corresponding to the second partition of the image block.
  • the initial current template is for the currently encoded image block.
  • the initial current template includes an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left edge of the image block. At least one of the adjacent areas.
  • the initial current template may include an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left edge of the image block; in another implementation , the initial current template may include an area adjacent to the upper boundary of the image block and an area adjacent to the left boundary of the image block.
  • the schematic diagram of the initial current template shown in Figure 5a includes the initial current template in the above two implementation modes.
  • the initial current template includes an area adjacent to the upper boundary of the image block and an area adjacent to the upper left corner of the image block. Other combinations of areas of the initial current template are not listed here.
  • dividing the initial current template based on the dividing line to obtain the current template corresponding to the first partition of the image block and the current template corresponding to the second partition of the image block may include the following: by The straight line where the dividing line used by the image block is located divides the initial current template into a current template corresponding to the first partition of the image block and a current template corresponding to the second partition of the image block.
  • the current template corresponding to the first partition of the image block and/or the current template corresponding to the second partition of the image block includes at least one boundary located on a straight line where the dividing line used by the image block is located. At least one boundary between the current template corresponding to the first partition and the current template corresponding to the second partition may be an adjacent edge of the current template corresponding to the two partitions.
  • the initial current template is divided according to the dividing line in GPM mode, and the partition template area can be obtained.
  • the partition template area includes the template area corresponding to each partition, that is, including the current template corresponding to the first partition (which can be called is the first current template) and the current template corresponding to the second partition (which can be called the second current template).
  • the relationship between the initial current template and the current templates of each partition may be: the initial current template includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the current template corresponding to the first partition and the current template corresponding to the second partition are adjacent, and the adjacent sides coincide with the straight line where the dividing line is located.
  • the current block includes the first partition and the second partition.
  • the current template corresponding to the first partition is the current template A
  • the current template corresponding to the second partition is the current template.
  • B the current template A and the current template B are adjacent, and the adjacent sides are located on the straight line where the dividing line is.
  • the difference between (1) and (2) in Figure 5b is that the areas included in the initial current template are different.
  • neither the current template A of the first partition nor the current template B of the second partition in example (2) of Figure 5b includes an adjacent area to the upper left of the image block.
  • the initial current template of example (1) in Figure 5b contains the area adjacent to the upper boundary of the image block, the area adjacent to the left boundary of the image block, and the adjacent area to the upper left of the image block, and the initial current template of (2) in Figure 5b only includes the area adjacent to the upper boundary of the image block and the area adjacent to the left boundary of the image block. Therefore, for different initial current templates, the current template A of the first partition and the current template B of the second partition include different areas.
  • the current template A of the first partition and the current template B of the second partition include different areas.
  • the current template A of the first partition includes an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an adjacent area to the upper left of the image block.
  • the current template B of the second partition includes the area adjacent to the upper boundary of the image block.
  • example (1) in Figure 5b is only used to illustrate the basic idea of the present invention.
  • the areas included by the current template A of the first partition and the current template B of the second partition are not fixed, but will vary according to different GPM division modes (that is, corresponding to different division lines).
  • the current template B of the second partition includes different areas.
  • the current template A of the first partition includes the area adjacent to the left boundary of the image block and a part of The upper left adjacent area of the image block
  • the current template B of the second partition includes the area adjacent to the upper boundary of the image block and a part of the upper left adjacent area of the image block
  • in example (2) of Figure 5b first
  • the current template A of the partition only includes the area adjacent to the left boundary of the image block
  • the current template B of the second partition only includes the area adjacent to the upper boundary of the image block.
  • the segmentation model is divided according to different GPMs.
  • the current module A and the current template B may include at least one of at least a part of the adjacent area directly above the current block, at least a part of the adjacent area to the left of the current block, and at least a part of the adjacent area above the left of the current block, or their combinations.
  • the current module A and the current template B may include at least a part of the adjacent area directly above the current block, and at least a part of the adjacent area to the left of the current block. At least one or a combination thereof.
  • the one between the current module A and the current template B includes at least part of the adjacent area directly above the current block and at least part of the adjacent area to the left of the current block, and the other one between the current module A and the current template B Only a portion of the adjacent area directly above the current block or at least a portion of the adjacent area to the left of the current block is included.
  • the other one between the current module A and the current template B only includes a part of the adjacent area directly above the current block or a part of the adjacent area to the left of the current block, or the other one between the current module A and the current template B only includes Includes the entire adjacent area directly above the current block or the entire adjacent area to the left of the current block.
  • the initial current template is divided according to the dividing lines in GPM mode, and the partitioned template area and the non-partitioned template area can be obtained.
  • the non-partition template area is a template area that does not serve as any partition and is an unusable area.
  • the straight line where the dividing line used for the image block passes through the non-partition template area, and the non-partition template area and the partition template area are adjacent.
  • the relationship between the initial current template and the current templates of each partition may be: the initial current template includes the current template corresponding to the first partition of the image block, the current template corresponding to the second partition of the image block, and the non-partition template. area.
  • some template areas in the initial current template cannot be used as the current template corresponding to the partition. At this time, the current template of the first partition and the current template corresponding to the second partition are not adjacent.
  • the current block includes a first partition and a second partition.
  • the current template corresponding to one partition is current template A
  • the current template corresponding to the second partition is current template B
  • current template A and current template B are not adjacent
  • the coordinate range of current template A and current template B (such as the abscissa range) They do not overlap.
  • Both current template A and current template B are adjacent to the unavailable area, and the unavailable area is a non-partition template. Similar to Figure 5b, the regions included in the initial current template in Example (1) and Example (2) in Figure 5c are different. That is, example (1) in Figure 5c includes the upper left adjacent area, while example (2) in Figure 5c does not include the upper left adjacent area. For details, please refer to the content introduced in Figure 5b and will not be described in detail here.
  • the current template corresponding to each partition in the image block shown in the aforementioned Figure 5b, Figure 5c, etc. is only an exemplary content. Due to the different types of dividing lines included in the geometric dividing mode, the type of the dividing lines Indicates how the dividing line divides the image blocks. The current template of each partition is determined based on the type of dividing line, and the area included in the initial current template also affects the final determination of the current template. Therefore, the current template corresponding to each partition is There are many types of templates, which are not shown one by one here, but they all fall within the scope of protection of this application.
  • the initial current template is not directly used as the template of the partition, but the current template corresponding to the partition is set for each partition. In this way, for a certain partition, the initial current template that is similar to another partition can be reduced. Some effects exist on template matching calculations. Therefore, the current template corresponding to each partition is set separately for each partition. Since the current template is a part similar to the corresponding partition, the template information of other partitions is not involved in the template matching calculation, and better matching can be achieved based on the current template. Generate similar templates to improve the template matching between similar templates and the current template, thereby improving the accuracy of prediction result calculations.
  • the type of the current template includes a beveled edge template and/or a non-beveled edge template.
  • the current template corresponding to each partition in the image block may include any one or both of a bevel template and a non-bevel template.
  • the current template of the first partition and the current template of the second partition both belong to hypotenuse templates, or both belong to non-hypotenuse templates.
  • the hypotenuse position and slope of the hypotenuse template are set according to the dividing lines dividing the image blocks.
  • the dividing line used in the image block is based on the fitted object boundary, setting the hypotenuse position and slope of the hypotenuse template through the dividing line can make the current template divided based on the dividing line have a high degree of fit to the image edge.
  • the hypotenuse boundary of the hypotenuse template coincides with the straight line where the dividing line is located.
  • the current templates shown in (1) in Figure 5b above are all hypotenuse templates, and the current template A or the current template B The hypotenuse coincides with the straight line where the dividing line lies.
  • the type of the current template is the non-hypotenuse template
  • the position of the non-hypotenuse template is set according to the area where the straight line of the dividing line passes through the initial current template.
  • the template area is neither the current template of the first partition nor the current template of the second partition, and belongs to the non-partition template area.
  • the current template A and the current template B both belong to the non-hypotenuse template, and the non-hypotenuse template is specifically a rectangular-shaped template. Since the non-hypotenuse template is a regular template, the calculation of the coordinates of the vertical line is smaller than that of the oblique line, so the calculation of the coordinates of the template corresponding to the partition can be effectively simplified.
  • the type of the similar template and the type of the current template may also be the same. For example, if the current template belongs to the hypotenuse template, then the similar template also belongs to the hypotenuse template. Only the position of the similar template and the current template in the image is different.
  • the current template also has the following characteristics: the abscissa value range of the current template is not equal to the length and/width of the image block; the ordinate value range of the current template is not equal to the length and/width of the image block.
  • the abscissa value range of the current template B shown in (1) in Figure 5b is smaller than the length of the image block, and the ordinate value range of the current template A is larger than the image block. of width. This is caused by the division of the dividing line.
  • the shape of the current template is set according to the position of the dividing line, which can avoid unnecessary sampling as a template, thereby making the content found according to this current template more similar to the partition. Thereby improving the accuracy of prediction results.
  • a similar template refers to a template that is similar to the current template.
  • the position of the similar template is different from the current template, but it has the same shape, size (specifically, the size of the image area), type, etc. as the current template. Therefore, similar templates can also have the following characteristics: the abscissa value range of the similar template is not equal to the length and/width of the image block; the ordinate value range of the similar template is not equal to the length and/width of the image block.
  • Any type of similar formwork includes beveled edge formwork and/or non-beveled edge formwork.
  • At least one similar template corresponding to the current template of each partition of the image block can be determined according to the preset strategy. That is, for the current template of each partition, at least one similar template corresponding to it can be determined. Due to the existence of the preset strategy, any partition in the image block can be matched with a rule that is more suitable for the partition to determine a similar template, thereby ensuring the accuracy of the similar template.
  • the implementation of S402 may include the following: determining at least one similar template corresponding to the current template from the predetermined search range according to the type of the geometric division pattern based on template matching.
  • Types of template matching-based geometric partitioning modes include template matching-based intra prediction and/or template matching-based inter prediction.
  • a predetermined search range is used to determine image regions of similar templates.
  • the predetermined search range may be a search range that matches the type of the geometric partitioning mode based on template matching.
  • the image block uses intra prediction based on template matching.
  • the image block is the currently encoded CU.
  • the predetermined search range may be the same as the CTU where the CU is located. Adjacent encoded CTU. It can be understood that when the target prediction mode is a geometric division mode based on template matching, for the current template corresponding to any partition in the image block, at least one corresponding to the current template of the partition can be determined in the manner described in S402. Similar templates.
  • the predetermined search ranges may be different (or the same).
  • corresponding predetermined search ranges can be matched, and at least one similar template corresponding to the current template can be determined therefrom, so that the similarity between the similar template and the current template is higher.
  • at least one similar template corresponding to the current template can be calculated based on the template cost.
  • the at least one similar template is a similar template among various similar templates that meets the matching requirements.
  • the matching requirement is, for example, that the template cost meets a cost threshold, or that the template cost is minimum.
  • the current template can correspond to one similar template, and the current template can also correspond to multiple (that is, at least two) similar templates.
  • the predetermined search range is determined based on a similar template corresponding to the reference block and/or the position of the prediction candidate block of the reference block.
  • the reference block is a coded block in the image where the image block is located or an image block in a reference image corresponding to the image where the image block is located.
  • the reference block is an image block in steps S301 to step S302 or a neighboring block of the image block in steps S401 to step S403, or the reference block is an image block in steps S301 to step S302.
  • the predetermined search range may be determined according to the similar template corresponding to the reference block and/or the position of the prediction candidate block of the reference block.
  • the predetermined search range can be determined based on the location of the similar template corresponding to the reference block, or the predetermined search range can be determined based on the location of the predicted candidate block of the reference block, or based on the location of the similar template corresponding to the reference block and the location of the reference block.
  • the positions of the predicted candidate blocks determine a predetermined search range.
  • the predetermined search range here is a search range corresponding to the currently encoded image block or a partition in the image block. In this search range, a similar template corresponding to the image block can be found, thereby determining the prediction candidate block of the image for use. Get prediction results based on prediction.
  • the similar templates, prediction candidate blocks, etc. corresponding to the reference block are all determinable data, and there is a correlation between the reference block and the currently encoded image block, the accuracy of determining the predetermined search range can be guaranteed based on these data, and Can improve the efficiency of determination.
  • determining the predetermined search range based on the position of the similar template corresponding to the reference block includes: determining the predetermined search range based on the positional relationship between the reference block and the image block and the position of the similar template corresponding to the reference block.
  • the positional relationship between the reference block and the image block includes an adjacent relationship (for example, left adjacent, upper adjacent), and the position of the similar template of the reference block can indicate the final predetermined search range of the current block. For example, if the reference block is the left adjacent block of the current block, a search range of a predetermined size can be determined on the right side of the similar template of the reference block as the predetermined search range of the current block.
  • a search range of a predetermined size is determined below the similar template of the reference block as the predetermined search range of the current block. Due to the correlation between the reference block and the image block, similar templates may also be related. It is a reference and convenient way to lock the predetermined search range of the currently encoded image block based on the position of the similar template of the reference block.
  • determining the predetermined search range according to the position of the prediction candidate block of the reference block includes: determining the predetermined search range according to the positional relationship between the reference block and the image block and the position of the prediction candidate block of the reference block.
  • the prediction candidate block of the reference block refers to the candidate block used for prediction corresponding to the reference block.
  • the position of the prediction candidate block of the reference block can be the position in the image where the reference block is located, or the position in the reference image corresponding to the image, depending on the geometric division based on template matching used by each partition in the reference block. The type of pattern.
  • the position of the prediction candidate block can be within the reconstructed image area of the image where the image block is located. If the type of the geometric division mode based on template matching is For inter-frame prediction based on template matching, the position of the prediction candidate block can be within the reference image corresponding to the image where the image block is located. In one embodiment, the positional relationship between the reference block and the image block includes an adjacent relationship. The predetermined search range can be determined by combining the positional relationship between different image blocks and the position of the prediction candidate block.
  • the reference block is a block adjacent to the left of the current block
  • determining the predetermined search range based on the position of the similar template corresponding to the reference block and the position of the prediction candidate block of the reference block includes: based on the positional relationship between the reference block and the image block, the prediction candidate of the reference block The position of the block and the position of the similar template corresponding to the reference block determine the predetermined search range.
  • the relevant explanations contained therein can be found in the foregoing description. For example, if the reference block is a block adjacent to the left of the current block, then a search range of a predetermined size is determined on the left side of the prediction candidate block and the right side of the similar template as The predetermined search range for the current block. In this way, the predetermined search range can be more accurately determined by integrating location information with rich related data.
  • the reference block may be a coded block in the image where the image block is located, such as an image block adjacent to the currently coded image block.
  • determining the predetermined search range based on the position of the final similar template and/or predicted candidate block of the reference block can greatly narrow the search range of similar templates and improve the search efficiency.
  • the reference block can also be an image block in the reference image corresponding to the image where the image block is located.
  • the reference image corresponding to the image where the image block is located can be an image that has been coded in the previous frame of the image where the image block is located.
  • the number of reference frames can include One or more, the positions of similar templates corresponding to the image blocks in the reference image can be used as reference data to determine the predetermined search range. In this way, there is no need to count the motion vector of the current block. When the motion vector of the image block is unavailable or When lost, this method can also be used as a backup method to quickly determine the predetermined search range based on known data and improve processing speed.
  • the size of the predetermined search range may be proportional to the size of the image block, so that each pixel has a fixed number of template cost comparison results, such as SAD (Sum of absolute differences, a matching Cost function, sum of absolute values of differences) comparison results.
  • SAD Sud of absolute differences, a matching Cost function, sum of absolute values of differences
  • the width of the predetermined search range a*BlkW
  • the height of the search range SearchRange_h a*BlkH, where a is a constant used to control the balance between gain and complexity. In practice, a can take 5, and BlkW and BlkH represent the width and height of the image block respectively. In this application, "*" means performing multiplication calculations.
  • the predetermined search range matches the type of the geometric division mode based on template matching: if the type of the geometric division mode based on template matching is inter prediction based on template matching, the predetermined search range The range is the image where the image block is located; if the type of the geometric division mode based on template matching is intra prediction based on template matching, the predetermined search range is the reference image corresponding to the image where the image block is located.
  • the predetermined search range matching the type of the geometric division mode based on template matching can be further narrowed, that is, in the image where the image block is located or in the image where the image block is located.
  • a smaller search range is locked in the corresponding reference image, and similar templates are determined from it. For example, if a partition in an image block uses the geometric partitioning mode of inter-frame prediction based on template matching, it can be determined that the predetermined search range is the coded area of the image where the image block is located.
  • the predetermined search range of the current block is determined again from the coded area. At this time, the predetermined search range is Narrowed search scope.
  • S403 Determine the prediction result of the image block according to the similar template.
  • the first prediction result set and the second prediction result set may be predictions based on the positioning of similar templates of the corresponding partitions.
  • the first prediction result set and the second prediction result set can be fused according to the weight value set of the partition to obtain the prediction result of the image block.
  • At least one of the first set of prediction results and the second set of prediction results may be determined through an intra prediction mode based on template matching.
  • the implementation of S403 includes the following: determining at least one prediction candidate block of the partition based on at least one similar template of the corresponding partition; determining the at least one prediction candidate block of each partition in the image block. The prediction result of the image block.
  • each partition of the image block corresponds to a current template
  • at least one similar template can be determined for the current template of each partition in the above manner. Therefore, the similar template can be, for the partition, the current template of each partition.
  • the number of corresponding similar templates may be the same or different.
  • the current template of the first partition corresponds to 3 similar templates
  • the current template of the second partition corresponds to 2 similar templates.
  • the corresponding partition can be any partition of the image block.
  • at least one prediction candidate block of the partition can be determined based on at least one similar template of any partition.
  • One similar template corresponds to one prediction candidate block.
  • three similar templates in the first partition can determine three prediction candidate blocks. Based on the adjacent relationship between the prediction candidate block and the similar template, the image area on the left or lower side adjacent to the similar template that has the same shape and size as the image block can be used as the prediction candidate block.
  • the prediction candidate block is A reconstructed image patch.
  • determining the prediction result of the image block based on the at least one prediction candidate block of each partition in the image block includes: weighting the at least one prediction candidate block of the corresponding partition according to a preset weight. , obtain the target prediction candidate block corresponding to the partition of the image block; determine the prediction result of the image block according to the partition weight and the target prediction candidate block of each partition in the image block.
  • At least one prediction candidate block For each partition in at least one partition of the image block, at least one prediction candidate block may be determined, and the at least one prediction candidate block of the partition may be weighted based on a preset weight, thereby obtaining the final prediction candidate block for the partition (i.e. target prediction candidate block).
  • Each prediction candidate block is configured with a preset weight, and the pixel values in each prediction candidate block can be multiplied and summed by the preset weight to obtain the target prediction candidate block.
  • at least one prediction candidate block includes prediction candidate block A1 and prediction candidate block A2. Using prediction candidate block A1 and prediction candidate block A2, the final prediction candidate block for partition A can be obtained, that is, :
  • Target prediction candidate block prediction candidate block A1*W1 + prediction candidate block A2*W2.
  • W1 and W2 are preset weights.
  • the final prediction candidate block for partition B can also be obtained by using the above method for at least one prediction candidate block corresponding to partition B (including prediction candidate block B1 and prediction candidate block B2).
  • integrating the known information in the reference image can make the prediction more accurate and the corresponding residual error smaller, which is beneficial to improving compression. efficiency.
  • each prediction candidate block is configured with a preset weight
  • the setting strategy of the preset weight includes at least one of the following: setting according to the distance between the prediction candidate block and the image block; setting according to the prediction Number of candidate blocks set; flags set updated based on weight.
  • the setting strategy of the preset weight includes: updating the flag setting according to the weight transmitted in the encoded image block.
  • the weight update flag wight_flag may be transmitted in the CTU or CU, and the weight update flag may indicate that the updated weight is adopted for the CTU or CU. For example, if the weight update flag is 1, it indicates that the CTU or CU adopts the updated weight, and/or if the weight update flag is 0, the weight is not updated.
  • the weight update flag does not exist in the bitstream, the default weight update flag is 0 and the CTU or CU does not adopt the updated weight. If the weight update flag is 1, the weight value is further analyzed at the decoding end.
  • the newly parsed weight value can be used for the current CTU or CU, or for the CTU or CU after the current CTU or CU encoding sequence, and can be used for the CU obtained by dividing the current CTU or CU using a quadtree.
  • the setting strategy of the preset weight includes: setting according to the distance between the prediction candidate block and the image block.
  • the preset weight of a prediction candidate block that is close to a partition in an image block may be greater than the preset weight of a prediction candidate block that is far from the partition.
  • the preset weights corresponding to different prediction candidate blocks may be unequal or different. equal. For example, if the preset weight W1 is 0.6 and W2 is 0.4, then the preset weight W1 is the weight of the prediction candidate block that is closer to the partition spatial location, and the default weight W2 is the prediction that is farther from the partition spatial location. The weight of the candidate block.
  • the distance between the corresponding prediction candidate block and the partition can also be determined. Normally, prediction candidate blocks that are closer to a partition are more similar to the partition. By assigning larger weights to prediction candidate blocks that are closer in spatial location, prediction accuracy can be improved.
  • the setting strategy of the preset weight includes: setting according to the number of prediction candidate blocks.
  • the number of prediction candidate blocks here refers to the number of at least one prediction candidate block corresponding to the current template.
  • the preset weight can be preset or updated based on the weight update flag.
  • the preset weights can be parsed based on the weight update flag. For example, when wight_flag is 1, the preset weights W1 and W2 are parsed, and the preset weights can be applied to the current CTU or CU.
  • the weight update flag wight_flag is 1, in the case of using equal numerical weights, the preset weight number wight_number can first be parsed. If the preset weight number wight_number is 2, then there are 2 preset weights, and each preset weight Both are 0.5. If the preset weight number wight_number is 4, then there are 4 preset weights, and each preset weight is 0.25. Furthermore, these preset weights can be used as the weights of the prediction candidate blocks of the current block.
  • the method of determining prediction results based on prediction candidate blocks as described above can also be directly used as a specific implementation method corresponding to S301 in the corresponding embodiment of Figure 3a. That is, the implementation of determining the prediction result of the image block according to the target prediction mode may be: determining the weighted prediction candidate corresponding to each partition in at least one partition of the image block according to the target prediction mode and the preset strategy. block to determine the prediction result of the image block.
  • the at least one prediction candidate block may be weighted according to the respective weights of the at least one prediction candidate block corresponding to each partition in the at least one partition of the image block to obtain the weighting process corresponding to each partition.
  • the final prediction candidate block is used to determine the prediction result of the image block; wherein the target prediction mode is used to determine the at least one partition, and the preset strategy is used to determine the corresponding corresponding value of the at least one partition.
  • the weight here is the same as the default weight introduced above.
  • the corresponding processing method can refer to the content introduced above and will not be described again here.
  • the image processing method provided by the embodiment of the present application can determine at least one similar template corresponding to the current template for each partition in the image block through a preset strategy when the target prediction mode includes a geometric division mode based on template matching, thereby based on the similar template Determine the prediction result of the image block.
  • the preset strategy can indicate the determination criteria of similar templates that match the current template of the image block
  • the preset strategy can be adapted to the geometric division mode based on template matching used in the corresponding partition to improve the relationship between similar templates and the current template.
  • the matching degree helps to improve the prediction accuracy.
  • each partition of the image block can also use the same or different types of geometric division modes based on template matching.
  • the preset strategy is specifically adapted to the type of the GPM mode based on template matching, and then the corresponding type of GPM mode based on template matching can be determined.
  • the template matches the similar template of the partition of the GPM pattern.
  • the similar template is a similar template with a better matching degree. More accurate prediction results can be obtained based on the similar template with a higher matching degree.
  • Figure 6 is a schematic flowchart of an image processing method according to a third embodiment.
  • the execution subject in this embodiment can be a computer device or a cluster composed of multiple computer devices.
  • the computer device can It may be an intelligent terminal (such as the aforementioned mobile terminal 100), or it may be a server.
  • the execution subject in this embodiment is an intelligent terminal as an example for explanation.
  • an optional implementation manner of determining at least one similar template corresponding to the current template from a predetermined search range according to the type of geometric division pattern based on template matching includes the following S601 and/or S602. It should be noted that for each partition in at least one partition of the image block (including the first partition and the second partition), the method as follows S601 or S602 can be used to determine at least one similarity corresponding to the current template of the partition. template.
  • S601 If the type of the geometric division mode based on template matching is intra prediction based on template matching, determine at least one similar template corresponding to the current template from the image where the image block is located.
  • the type of the geometric division mode based on template matching used in the partition is intra prediction based on template matching
  • the partition using the geometric division mode based on template matching can refer to the processing method of intra prediction.
  • at least one similar template corresponding to the current template of the partition can be determined from the image where the image block is located.
  • Intra prediction based on template matching is a special intra prediction mode, which can be referred to as template intra prediction for short.
  • Template intra prediction can determine similar templates corresponding to the current template of the partition in the current image.
  • Intra prediction based on template matching copies the prediction candidate block from the reconstructed part of the current image (that is, the image where the current block is located), and the similar template of the prediction candidate block matches the current template.
  • the best similar template of the current template may be searched, and the best similar template may be used as the similar template corresponding to the current template using the partition based on template matching for intra prediction.
  • the encoder may send a flag indicating the use of template matching based intra prediction. The usage here refers to which partition of the image block is used by intra prediction based on template matching.
  • the decoder will parse this flag to decide whether to enable intra prediction based on template matching.
  • the first partition of the image block uses template matching-based intra prediction, but the second partition is not used.
  • Intra prediction with template matching is applied to the first partition of the image patch.
  • the implementation of S601 includes: determining a similar template that matches the current template from the reconstructed image area in the image where the image block is located according to preset search rules; determining the said template based on the matched similar template. At least one similar template corresponding to the current template.
  • the reconstructed image area in the image where the image block is located is a coded area, and the reconstructed image area can be used as a predetermined search range, from which at least one similar template is searched, and at least one similar template is the best similar template of the current template.
  • the reconstructed image area in the image where the image block is shown in Figure 7a including R1, R2, R3, and R4.
  • R1 is the reconstruction area within the current CTU adjacent to the upper left corner of the image block CU
  • R2 is the reconstruction area adjacent to the upper left corner of the current CTU
  • R3 is the reconstruction area adjacent to the upper boundary of the CTU
  • R4 is the left boundary of the current CTU. adjacent redevelopment area.
  • a similar template A1 corresponding to the current template A of the first partition is determined in the reconstruction area R1
  • a similar template A2 matching the current template A of the first partition is determined in the reconstruction area R2
  • a similar template B1 matching the current template B of the second partition is determined in the reconstruction area R3
  • a similar template B2 matching the current template B of the second partition is determined in the reconstruction area R2.
  • the preset search rule refers to a rule for searching for similar templates that match the current template in the reconstructed image area.
  • the preset search rule is, for example, to stop the search when a specified number of similar templates are found, or to specify a search method.
  • the reconstructed image area in the image where the image block is located can be Regularly search for similar templates that match the current template, and can be based on specific content settings of preset search rules, which can improve search efficiency and/or search comprehensiveness to a certain extent.
  • the preset search rules include at least one of the following: 1 During the search for the reconstructed image area in the image where the image block is located, when a preset number of targets that meet the matching conditions appear area, then determine each target area that meets the matching conditions as a similar template that matches the current template; 2 After the search for the reconstructed image area is completed, determine a preset number of target areas that meet the matching conditions as matching the current template. Similar templates.
  • the preset search rule 1 during the search process of the reconstructed image area, when the first target area that meets the matching conditions appears, this area can be used as a similar template that matches the current template. When the second target area that meets the matching conditions appears, The target area can also be used as a similar template matching the current template.
  • These target areas are reconstruction blocks with the same shape and size as the currently encoded image block. In this way, when a preset number of target areas that meet the matching conditions appear, regardless of whether the reconstructed image area is searched or not, the search can be stopped, and each target area that meets the matching conditions is obtained as each similar template for the current template matching. In this way, the search for similar templates may be able to determine the similar template that matches the current template without searching the reconstructed image area.
  • the search efficiency is high and the search requires few resources. Since the matched similar template satisfies the matching conditions, It can improve the overall coding efficiency while ensuring the accuracy of subsequent prediction results.
  • the preset search rule 2 during the search process of the reconstructed image area, the matching degree of each area and the current block can be recorded. After the search of the reconstructed image area is completed, the recorded matching degree of each area can satisfy the matching conditions.
  • a preset number of target areas are determined as similar templates matching the current template. For example, after searching the reconstructed image area, two (the default number is 2) target areas that meet the matching conditions appear, then these two target areas can be used as similar templates for current template matching. This enables a complete and comprehensive search of the entire reconstructed image area, and determines similar templates that match the current template through comprehensive comparison, which is beneficial to providing similar templates with high matching degrees, thus helping to improve the accuracy of prediction results.
  • the preset number is a limit on the number of similar templates that match the current template, which can be 1, 2, or 3, etc., and is not limited here.
  • Meeting the matching conditions can mean that the matching degree between the target area and the image block meets the requirements.
  • the matching degree can be measured by the template cost between the target area and the image block.
  • the matching degree can mean that the template cost is the smallest, or the template cost is the smallest.
  • the cost is less than or equal to the template cost threshold.
  • the template cost can be the SSD (Sum of Squared Difference, sum of squares of differences) between the current template and the searched template, or it can be SAD (Sum of Absolute Difference, sum of absolute errors), or other Calculation function.
  • a small template cost indicates a high matching degree and a large similarity between the searched template and the current template.
  • a large template cost indicates a low matching degree and a small similarity between the searched template and the current template.
  • a similar template can be searched within one or more predetermined ranges so that the SAD between the similar template and the current template is the smallest.
  • the SSD between the region and the current template can be calculated.
  • the target region with an SSD smaller than the threshold can be used as a similar template that matches the current template, or the search region can be The smallest target area of SSD serves as a similar template that matches the current template.
  • the degree of similarity between the region and the current template can be determined by the size of the SSD value. If the SSD is less than a threshold, the region can be considered to be a similar template that matches the current template.
  • the degree of similarity between the current template and the similar template is relatively small. high.
  • determining at least one similar template corresponding to the current template based on the matched similar template includes the following: determining a template cost between the matched similar template and the current template, which will satisfy a preset
  • the matched similar template corresponding to the template cost of the template condition is determined to be at least one similar template corresponding to the current template.
  • a template cost between the matched similar template and the current template may be calculated.
  • the template cost can represent the rate distortion cost or the similarity between templates.
  • the similarity between templates can be measured by the mean square and SSD of the difference between the matched similar template and the current template. It can also be measured by, for example, the absolute error. and SAD, or measured in other ways.
  • satisfying the preset template condition includes at least one of the following: the template cost is minimum, and the template cost is less than or equal to the preset cost threshold. The preset cost threshold is used to determine whether the matched similar template is the best similar template of the current template.
  • the template cost is the smallest, which means that the matched similar template has the highest matching degree with the current template. Using it as the best similar template can improve the accuracy of the prediction results.
  • the template cost is less than or equal to the preset cost threshold, indicating that the matched similar template has a high degree of matching with the current template, and can also be used as the best similar template of the current template. In this case, the number of best similar templates may include multiple , more information is referenced in the process of determining the prediction results.
  • the similar template corresponding to the minimum template cost that matches the current template can be determined as the final similar template corresponding to the current template, or the similar template corresponding to the template cost that is less than or equal to the preset cost threshold and matched with the current template can be determined as the current template.
  • the template cost threshold can also be used to select similar templates that match the current template.
  • the preset set here Let the cost threshold be greater than the aforementioned template cost threshold.
  • the matched similar templates from the initial screening can be further screened, and whether the similarity template meets the preset template conditions can be judged again based on the template cost, so as to determine the similar template with a higher matching degree.
  • Improve the similarity between similar templates and the current template reduce the residual error in the prediction process, and then obtain prediction results that more accurately restore the input original image blocks.
  • similar templates may be used to determine predictions for image blocks.
  • the implementation may be as described in S403 in the embodiment of Figure 4a.
  • the prediction candidate block can be determined based on the similar template, from which a prediction candidate block can be determined directly as the prediction result of the image block, or at least one prediction candidate block can be determined from it After performing the weighted summation, the prediction result of the image block is determined.
  • the current template here is the current template corresponding to the partition using intra prediction based on template matching in the image block.
  • the current template is a non-hypotenuse template, and both the first partition and the second partition use intra prediction based on template matching.
  • Figure 7b shows a schematic diagram of intra prediction based on template matching. As shown in Figure 7b, the similar template A1 and similar template A2 corresponding to the current template A are determined in the reconstructed image area of the image where the current block is located, and the similar template B1 and similar template B1 corresponding to the current template B are determined.
  • Corresponding prediction candidate blocks are determined, including prediction candidate block A11, prediction candidate block A22, prediction candidate block B11, and prediction candidate block B22, and then the prediction result of the image block can be further determined based on these prediction candidate blocks.
  • S602 If the type of the geometric division mode based on template matching is inter-frame prediction based on template matching, determine at least one similar template corresponding to the current template from the reference image corresponding to the image where the image block is located.
  • the type of the geometric partition mode based on template matching used in the partition is inter-frame prediction based on template matching
  • the partition using the geometric partition mode based on template matching can refer to the processing method of inter-frame prediction.
  • at least one similar template corresponding to the current template of the partition can be determined from the reference image corresponding to the image where the image block is located.
  • S602 includes the following method 1 and method 2:
  • Method 1 Determine at least one initial similar template from the reference image corresponding to the image where the image block is located based on at least one initial motion vector, and search for a similar template that matches the current template within a predetermined range including any one of the initial similar templates. , determining at least one similar template corresponding to the current template according to the matched similar templates.
  • the similar template corresponding to the current template can be determined in the reference image based on the motion vector.
  • one or more motion vectors can be selected from the motion vector candidate list as the initial motion vector.
  • an initial similar template can be determined based on the selected initial motion vector.
  • a predetermined range including the initial similar template can be searched.
  • the matched similar template is the best similar template. It can be directly determined as a similar template corresponding to the current block, and the corresponding template is determined based on the position of the similar template. Predict candidate blocks.
  • Partition A adopts the inter-frame prediction based on template matching. Based on the initial motion vector MV1, a similar template that best matches the current template A is searched in the reference image 1. A1, and determine the prediction candidate block A11 based on the similar template A1.
  • multiple motion vectors can also be selected from the motion vector list as initial motion vectors mv1, mv2, ..., mvn (n is greater than or equal to 2).
  • n initial similar templates are determined respectively according to the initial motion vectors mv1, mv2,..., mvn.
  • these similar templates are better similar templates.
  • the best similar template is determined among the matched n similar templates, and The best similar template is used as the similar template corresponding to the current template.
  • Figure 7d is a schematic diagram of adjacent block positions of the motion vector candidate list
  • Figure 7e is a motion vector candidate list.
  • the spatial merging candidate list can select up to 4 candidate motion vectors.
  • the construction order is: upper adjacent block B1, left adjacent block A1, upper right adjacent block B0, lower left adjacent block A0, upper left adjacent block In block B2, the motion vectors of the block Col corresponding to the reference image are arranged in sequence, and these motion vectors are used as motion vector candidates of the current block. It should be noted that B2 will be considered only when other locations are unavailable.
  • the historical reference block his, the average motion vector avg of the first candidate motion vector, the second candidate motion vector, and the zero motion vector 0 can also be added to the merge candidate list.
  • the reference image includes a first reference image and/or a second reference image, and the first similar template corresponding to the current template is determined from the first reference image corresponding to the image where the image block is located, and/or, from Determine a second similar template corresponding to the current template in the second reference image corresponding to the image where the image block is located, and determine the first similar template and/or the second similar template as at least one corresponding to the current template. Similar templates.
  • the reference image may include a first reference image and a second reference image.
  • the similar template may be determined in two frames of reference images before and after the image in which the current block is located, or in one frame of reference image. That is, the first similar template corresponding to the current template can be determined from the first reference image and the second similar template corresponding to the current template can be determined from the second reference image, and the first similar template and the second similar template are at least the same as the current template.
  • the first similar template may also be determined only from the first reference image, and the first similar template may be used as at least one similar template corresponding to the current template.
  • the second similar template may also be determined only from the second reference image, and the second similar template may be used as at least one similar template corresponding to the current template.
  • the frame type of the image where the image block is located may be an I frame or a P frame, and a similar template is determined from the first reference image and/or the second reference image based on different frame types. For example, if the image block is a P Slice (unidirectional inter-frame coding slice), it can be determined from a reference image, for example, at least one similar template corresponding to the current template is determined from the first reference image or the second reference image. .
  • P Slice unidirectional inter-frame coding slice
  • the image block is a B Slice (bidirectional inter-frame coded slice)
  • at least one similar template corresponding to the current template can be determined from two reference images, including the first reference image and the second reference image.
  • relevant resources can be more effectively utilized to obtain at least one similar template corresponding to the current template, and efficient compression or decompression of image blocks can be achieved.
  • the first similar template (second similar template) corresponding to the current template can be determined from the first reference image (or the second reference image) corresponding to the image where the image block is located, as in method 1.
  • the content described is to determine similar templates through motion vectors.
  • Partition B uses inter-frame prediction based on template matching.
  • the similar template B1 corresponding to the current template B can be determined in the reference image 1 based on the initial motion vector MV1, and the prediction candidate block B11 is determined based on the similar template B1.
  • Based on the initial motion vector MV2 determines the similar template B2 corresponding to the current template B in the reference image 2, and determines the prediction candidate block B22 based on the similar template B2.
  • each partition can determine at least one similar template of the corresponding partition according to the method of S601; for example, both the first partition and the second partition adopt inter-frame prediction based on template matching, then at least one similar template of the corresponding partition can be determined according to the method of S602. .
  • each partition uses inter-frame prediction based on template matching.
  • the two partitions of the current block in the current image both use inter-frame prediction based on template matching, based on the initial motion vector MV1.
  • the similar template A1 corresponding to the current template A can be determined in the reference image 1, and the prediction candidate block A11 can be determined based on the similar template A1, and the similar template B1 corresponding to the current template B can be determined in the reference image 2 based on the initial motion vector MV2. , and based on the similar template B1
  • the prediction candidate block B11 is determined, and then the prediction result of the current block is determined based on the prediction candidate block A11 and the prediction candidate block B11.
  • each partition in the image block corresponds to at least one similar template in the current template through a preset strategy.
  • the first partition and the second partition in the image block adopt different types of geometric division modes based on template matching, for example, the first partition adopts intra prediction based on template matching, and the second partition adopts intra prediction based on template matching.
  • Inter-frame prediction the first partition can determine at least one similar template of the first partition according to S601, and the second partition can determine at least one similar template of the second partition according to S601.
  • the first partition can determine at least one similar template of the first partition according to the method of S602
  • the second partition can determine Determine at least one similar template of the second partition according to the method of S601.
  • partition B of the current block in the current image uses inter prediction based on template matching
  • partition A uses intra prediction based on template matching
  • the inter prediction based on template matching used in partition B of the current block is a bidirectional prediction mode. In this mode, there are two motion vectors in partition B, namely the initial motion vector MV1 and the initial motion vector MV2. Based on the initial motion vector MV1, the similar template B1 corresponding to the current template B is determined in the reference image 1. Based on the initial motion vector MV2, the similar template B2 corresponding to the current template B is determined in the reference image 2.
  • Similar template B2 and similar template B1 are both At least one similar template can be used as the current template, and then the prediction candidate block B11 is determined based on the similar template B1, and the prediction candidate block B22 is determined based on the similar template B2.
  • the prediction candidate blocks B11 and B22 can be further weighted and summed to obtain the target corresponding to partition B. prediction block.
  • Partition A uses intra prediction based on template matching. In the reconstruction area adjacent to the current CTU, similar template A1 and similar template A2 corresponding to the current template A are searched, and then the prediction candidate block A11 is determined based on the similar template A1, and Based on the similar template A2, the prediction candidate block A21 is determined.
  • inter prediction based on template matching may be a unidirectional prediction mode. That is to say, if a partition adopts unidirectional prediction mode, there is only one motion vector in this partition.
  • partition A of the current block in the current image may adopt inter prediction based on template matching
  • partition B may adopt intra prediction based on template matching.
  • each partition of the image block can flexibly combine different or the same type of template matching-based GPM mode, enriching the prediction processing method for the same image block, making the prediction more flexible.
  • the types of GPM patterns based on template matching used in each partition of the image block are different, not only the matching similar template can be searched in the reference image, but also the matching similar template can be searched in the coded area of the current image, so that the matching similar template can be better Searching for the best matching similar template will help improve the accuracy of the prediction results of image blocks, reduce image distortion, and better restore the real image.
  • the image processing method provided by the embodiment of the present application adopts different prediction processing methods for different types of geometric division patterns based on template matching to obtain at least one similar template corresponding to the current template.
  • This prediction processing method matches the type of geometric division mode based on template matching, so that similar templates with better matching can be better searched for each partition. Similar templates with better matching can provide prediction results. A more accurate basis for processing.
  • Figure 8 is a schematic flow chart of an image processing method according to a fourth embodiment.
  • the execution subject in this embodiment can be a computer device or a cluster composed of multiple computer devices.
  • the computer device can It may be an intelligent terminal (such as the aforementioned mobile terminal 100), or it may be a server.
  • the execution subject in this embodiment is an intelligent terminal as an example for explanation.
  • the implementation of determining the prediction result of the image block according to the target prediction mode may include the content described in S802.
  • the implementation of determining the prediction result of the image block according to the target prediction mode and the preset strategy may include the content described in S802.
  • S802 Determine the initial current template of the image block, so as to determine the prediction result of the image block through the initial current template.
  • the initial current template of an image block includes the coded image region adjacent to the image block.
  • the initial current template includes a partitioned template area and a non-partitioned template area.
  • the partition template area includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the initial current template includes an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left edge of the image block. at least one area in; the initial current template includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the initial current template shown in Figure 5a please refer to the initial current template shown in Figure 5a.
  • the current template corresponding to each partition has the following characteristics: the abscissa range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition; or, the current template corresponding to the first partition
  • the ordinate range is different from the ordinate range of the current template corresponding to the second partition; or, the length of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition is different from the length of the current template corresponding to the second partition.
  • the lengths of image blocks and/or prediction candidate blocks of the image blocks are not equal; or, the width of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition is different from the length of the image block. and/or the prediction candidate blocks of the image block have different widths.
  • the current template corresponding to the first partition of the image block and/or the current template corresponding to the second partition of the image block includes at least one boundary, and the at least one boundary is located at the location used by the image block. on the straight line where the dividing line is located; or, the initial current template includes the current template corresponding to the first partition of the image block, the current template corresponding to the second partition of the image block, and the non-partition template area, and the image The straight line where the dividing line used by the block passes through the non-partition template area; or the current template corresponding to the non-partition template area and the first partition of the image block and the current template corresponding to the second partition of the image block at least one adjacent.
  • the current template of each partition shown in Figure 5b For a detailed introduction to the above content, please refer to the relevant content in the corresponding embodiment of Figure 4a.
  • the prediction result of the image block can be determined through the initial current template.
  • the current template corresponding to each partition can be obtained by dividing the initial current template of the image block.
  • the current template corresponding to each partition can be obtained by dividing the initial current template of the image block.
  • the straight line where the dividing line used by the image block is located divides the initial current template into a current template corresponding to the first partition of the image block and a current template corresponding to the second partition of the image block; wherein, the image block
  • the current template corresponding to the first partition and/or the current template corresponding to the second partition of the image block includes at least one boundary located on a straight line where the dividing line used by the image block is located.
  • At least one similar template corresponding to the current template of the partition can be determined from the image where the image block is located, or from the reference image corresponding to the image where the image block is located.
  • a similar template used for matching in the prediction process can first be determined, so that the prediction result can be more accurately determined using the initial current template.
  • the implementation of determining the prediction result of the image block according to the target prediction mode may also adopt the content described in S803 below.
  • the implementation of determining the prediction result of the image block according to the target prediction mode and the preset strategy may include the content described in S803.
  • S803 Determine the weighted prediction candidate block corresponding to each partition in at least one partition of the image block according to the target prediction mode and the preset strategy, so as to determine the prediction result of the image block.
  • the target prediction mode includes a geometric partitioning mode based on template matching.
  • the image block is divided into at least one partition, including a first partition and a second partition.
  • the weighted prediction candidate blocks of each partition of the image block can be determined, and then the prediction result of the image block is determined based on the prediction candidate blocks.
  • the preset strategy is used to indicate the criteria for template matching in the geometric division mode based on template matching. Through this preset strategy, similar templates that match the current template of the image block can be found.
  • determining the prediction result of the image block according to the target prediction mode may include the following implementation steps: according to the target prediction mode and the preset strategy, determine the corresponding response of each partition in at least one partition of the image block. At least one prediction candidate block; perform weighting processing on the at least one prediction candidate block to obtain a weighted prediction candidate block corresponding to each partition, which is used to determine the prediction result of the image block.
  • each prediction candidate block can be determined, and each prediction candidate block can be configured with a preset weight.
  • each prediction candidate block can be weighted based on the preset weight to obtain a weighted prediction candidate block of the partition.
  • it can be called a target prediction candidate block.
  • Each partition corresponds to a target prediction candidate block.
  • the partition of the image block includes a first partition and a second partition. The prediction result of the image block can be determined based on the target prediction candidate block of the first partition and the prediction candidate block of the second partition. .
  • the process of determining the prediction candidate block includes: first determining the initial current template of the image block, and determining the current template of each partition of the image block based on the initial current template, and then determining the current template corresponding to each partition. Multiple similar templates, and finally at least one prediction candidate block is determined based on multiple similar templates, and one similar template can determine one prediction candidate block.
  • the prediction candidate blocks of the image blocks can be flexibly and accurately determined based on the target prediction mode and the preset strategy, thereby helping to improve the accuracy of the prediction results of the image blocks.
  • step S802 can be combined with S801 to form a solution for determining the prediction result of the image block
  • step S803 can also be combined with S801 to form a solution for determining the prediction result of the image block.
  • These two plans can be parallel plans.
  • Figure 9 is a schematic structural diagram of an image processing device according to an embodiment of the present application.
  • the image processing device can be a computer program (including program code) running in a server.
  • the image processing device is an application.
  • Software this device can be used to execute corresponding steps in the method provided by the embodiments of this application.
  • the image processing device 900 includes: an acquisition module 901 and a determination module 902.
  • Acquisition module 901 used to obtain or determine the target prediction mode of the image block
  • Determining module 902 configured to determine the prediction result of the image block according to the target prediction mode.
  • the target prediction mode includes a template matching-based geometric division mode, and the image block is divided into a first partition and a second partition under the template matching-based geometric division mode.
  • the type of the geometric partitioning mode based on template matching includes: intra prediction based on template matching and/or inter prediction based on template matching; the first partition adopts the geometric partitioning mode based on template matching. Intra prediction based on template matching or inter prediction based on template matching; the second partition adopts the intra prediction based on template matching or the inter prediction based on template matching.
  • the target prediction mode includes a geometric division mode based on template matching.
  • the determination module 902 is configured to: determine through a preset strategy that each partition in the image block corresponds to at least one similar template in the current template; according to The similarity template determines the prediction result of the image block.
  • the determination module 902 is configured to determine at least one similar template corresponding to the current template from a predetermined search range according to the type of the geometric division pattern based on template matching.
  • the predetermined search range is determined based on a similar template corresponding to the reference block and/or the position of the prediction candidate block of the reference block.
  • the determination module 902 is further configured to: determine the predetermined search range according to the positional relationship between the reference block and the image block and the position of the similar template corresponding to the reference block; and/ Or, the predetermined search range is determined based on the positional relationship between the reference block and the image block and the position of the prediction candidate block of the reference block.
  • the determining module 902 is configured to: if the type of the template matching-based geometric division mode is template matching-based intra prediction, then Determine at least one similar template corresponding to the current template from the image in which the image block is located; and/or, if the type of the geometric division mode based on template matching is inter-frame prediction based on template matching, determine from the image block Determine at least one similar template corresponding to the current template in the reference image corresponding to the image.
  • the determination module 902 is configured to: determine a similar template that matches the current template from the reconstructed image area in the image where the image block is located according to preset search rules; determine the corresponding template based on the matched similar template. At least one similar template corresponding to the current template.
  • the determining module 902 is configured to: determine the first similar template corresponding to the current template from the first reference image corresponding to the image where the image block is located, and/or, from the image where the image block is located Determine a second similar template corresponding to the current template in the corresponding second reference image, and determine the first similar template and/or the second similar template as at least one similar template corresponding to the current template; and/or, Determine at least one initial similar template from the reference image corresponding to the image where the image block is located based on at least one initial motion vector, search for a similar template that matches the current template within a predetermined range including any one of the initial similar templates, according to the The matching similar templates determine at least one similar template corresponding to the current template.
  • a determination module 902 configured to: during the process of searching for the reconstructed image area in the image where the image block is located, when a preset number of target areas that meet the matching conditions appear. , then determine each target area that meets the matching conditions as a similar template that matches the current template; and/or, after the search for the reconstructed image area is completed, determine a preset number of target areas that meet the matching conditions as similar templates that match the current template. Matching similar templates.
  • the determination module 902 is configured to: determine the template cost between the matched similar template and the current template, and determine the matched similar template corresponding to the template cost that satisfies the preset template condition as the current template. Corresponding to at least one similar template.
  • the determination module 902 is configured to: determine at least one prediction candidate block of the partition based on at least one similar template of the corresponding partition; determine the at least one prediction candidate block of each partition in the image block. Prediction results of image blocks.
  • the determination module 902 is configured to: obtain the initial current template of the image block; divide the initial current template into the current template corresponding to the first partition of the image block and/or the image block The second partition corresponds to the current template.
  • the acquisition module 901 is configured to at least one of the following: determine the target prediction mode of the image block through the first flag; determine the prediction mode corresponding to the minimum rate distortion cost as the target prediction mode of the image block .
  • a first flag is used to indicate that the image block uses the geometric division mode based on template matching, and using at least one second flag to indicate the prediction processing used by each of the first partition and/or the second partition of the image block.
  • the determination module 902 is configured to: determine a weighted prediction candidate block corresponding to each partition in at least one partition of the image block according to the target prediction mode and the preset strategy, so as to Used to determine the prediction result of the image block.
  • the determining module 902 is configured to: determine an initial current template of the image block for determining the initial current template.
  • the partition template area includes a first current template corresponding to the first partition and a second current template corresponding to the second partition.
  • the initial current template includes at least one of an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left side of the image block. ;
  • the initial current template includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the abscissa range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition; or, the ordinate range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition.
  • the ordinate range of the current template corresponding to the second partition is different; or, the length of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition is different from the image block and/or the length of the current template corresponding to the second partition.
  • the lengths of the prediction candidate blocks of the image block are not equal; or, the width of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition are different from those of the image block and/or the image.
  • the prediction candidate blocks of the blocks are not equal in width.
  • the current template corresponding to the first partition of the image block and/or the current template corresponding to the second partition of the image block includes at least one boundary, and the at least one boundary is located on the dividing line used by the image block. on the straight line; or,
  • the initial current template includes a current template corresponding to the first partition of the image block, a current template corresponding to the second partition of the image block, and a non-partition template area, where the straight line where the dividing line used by the image block is located passing through the non-partitioned template area; or the non-partitioned template area is adjacent to at least one of the current template corresponding to the first partition of the image block and the current template corresponding to the second partition of the image block.
  • the type of the current template includes a beveled edge template and/or a non-beveled edge template, and the type of the at least one similar template is the same as the type of the current template; when the type of the current template is the beveled edge template When the edge template is a side template, the hypotenuse position and slope of the hypotenuse template are set according to the dividing line dividing the image block; when the type of the current template is the non-hypotenuse template, the position of the non-hypotenuse template Set according to the area where the straight line of the dividing line passes through the initial current template.
  • An embodiment of the present application also provides an intelligent terminal, which includes a memory and a processor.
  • An image processing program is stored in the memory. When the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.
  • the smart terminal may be the mobile terminal 100 as shown in FIG. 1 .
  • the processor 110 of the mobile terminal 100 as shown in Figure 1 can be used to call the image processing program stored in the memory 109 to perform the following operations: S1: Obtain or determine the target prediction mode of the image block ; S2: Determine the prediction result of the image block according to the target prediction mode.
  • the target prediction mode includes a template matching-based geometric division mode, and the image block is divided into a first partition and a second partition under the template matching-based geometric division mode.
  • the type of the geometric partitioning mode based on template matching includes: intra prediction based on template matching and/or inter prediction based on template matching; the first partition adopts the geometric partitioning mode based on template matching. Intra prediction based on template matching or inter prediction based on template matching; the second partition adopts the intra prediction based on template matching or the inter prediction based on template matching.
  • the target prediction mode includes a geometric division mode based on template matching.
  • the processor 110 is configured to: determine through a preset strategy that each partition in the image block corresponds to at least one similar template in the current template; according to The similarity template determines the prediction result of the image block.
  • the processor 110 is configured to determine at least one similar template corresponding to the current template from a predetermined search range according to the type of the geometric division pattern based on template matching.
  • the predetermined search range is determined based on a similar template corresponding to the reference block and/or the position of the prediction candidate block of the reference block.
  • the processor 110 is further configured to: determine the predetermined search range based on the positional relationship between the reference block and the image block and the position of the similar template corresponding to the reference block; and/ Or, the predetermined search range is determined based on the positional relationship between the reference block and the image block and the position of the prediction candidate block of the reference block.
  • the processor 110 is configured to: if the type of the geometric division mode based on template matching is intra prediction based on template matching, determine at least one corresponding to the current template from the image where the image block is located. A similar template; and/or, if the type of the geometric division mode based on template matching is inter-frame prediction based on template matching, determine at least one corresponding to the current template from the reference image corresponding to the image where the image block is located Similar templates.
  • the processor 110 is configured to: determine a similar template that matches the current template from the reconstructed image area in the image where the image block is located according to preset search rules; determine the corresponding template based on the matched similar template. At least one similar template corresponding to the current template.
  • the processor 110 is configured to: determine the first similar template corresponding to the current template from the first reference image corresponding to the image where the image block is located, and/or, from the image where the image block is located Determine a second similar template corresponding to the current template in the corresponding second reference image, and determine the first similar template and/or the second similar template as at least one similar template corresponding to the current template; according to at least one initial The motion vector determines at least one initial similar template from the reference image corresponding to the image where the image block is located, searches for a similar template that matches the current template within a predetermined range including any one of the initial similar templates, and based on the matched similar template Determine at least one similar template corresponding to the current template.
  • the processor 110 is configured to: during the process of searching for the reconstructed image area in the image where the image block is located, when a preset number of target areas that meet the matching conditions appear. , then determine each target area that meets the matching conditions as a similar template that matches the current template; and/or, after the search for the reconstructed image area is completed, determine a preset number of target areas that meet the matching conditions as similar templates that match the current template. Matching similar templates.
  • the processor 110 is configured to: determine the template cost between the matched similar template and the current template, and determine the matched similar template corresponding to the template cost that satisfies the preset template condition as the current template. Corresponding to at least one similar template.
  • the processor 110 is configured to: determine at least one prediction candidate block of the partition based on at least one similar template of the corresponding partition; determine the at least one prediction candidate block of each partition in the image block. Prediction results of image blocks.
  • the processor 110 is configured to: obtain an initial current template of the image block; divide the initial current template into a current template corresponding to the first partition of the image block and/or the image block The second partition corresponds to the current template.
  • the processor 110 is configured to at least one of the following: determine the target prediction mode of the image block through the first flag; determine the prediction mode corresponding to the minimum rate distortion cost as the target prediction mode of the image block .
  • a first flag is used to indicate that the image block uses the geometric division mode based on template matching, and using at least one second flag to indicate the prediction processing used by each of the first partition and/or the second partition of the image block.
  • the processor 110 is configured to: determine a weighted prediction candidate block corresponding to each partition in at least one partition of the image block according to the target prediction mode and the preset strategy, so as to Used to determine the prediction result of the image block.
  • the processor 110 is configured to: determine an initial current template of the image block for determining the initial current template.
  • the partition template area includes a first current template corresponding to the first partition and a second current template corresponding to the second partition.
  • the initial current template includes at least one of an area adjacent to the upper boundary of the image block, an area adjacent to the left boundary of the image block, and an area adjacent to the upper left side of the image block. ;
  • the initial current template includes the current template corresponding to the first partition and the current template corresponding to the second partition.
  • the abscissa range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition; or, the ordinate range of the current template corresponding to the first partition is different from the abscissa range of the current template corresponding to the second partition.
  • the ordinate range of the current template corresponding to the second partition is different; or, the length of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition is different from that of the image block and/or the length of the current template corresponding to the second partition.
  • the lengths of the prediction candidate blocks of the image block are not equal; or, the width of the current template corresponding to the first partition and/or the length of the current template corresponding to the second partition are different from those of the image block and/or the image.
  • the prediction candidate blocks of the blocks are not equal in width.
  • the current template corresponding to the first partition of the image block and/or the current template corresponding to the second partition of the image block includes at least one boundary, and the at least one boundary is located on the dividing line used by the image block. on the straight line; or,
  • the initial current template includes a current template corresponding to the first partition of the image block, a current template corresponding to the second partition of the image block, and a non-partition template area, where the straight line where the dividing line used by the image block is located Passing through the non-partition template area; or between the non-partition template area and the image block
  • the current template corresponding to the first partition is adjacent to at least one of the current templates corresponding to the second partition of the image block.
  • the type of the current template includes a beveled edge template and/or a non-beveled edge template, and the type of the at least one similar template is the same as the type of the current template; when the type of the current template is the beveled edge template When the edge template is a side template, the hypotenuse position and slope of the hypotenuse template are set according to the dividing line dividing the image block; when the type of the current template is the non-hypotenuse template, the position of the non-hypotenuse template Set according to the area where the straight line of the dividing line passes through the initial current template.
  • the mobile terminal described in the embodiments of this application can execute the method description in any of the above embodiments, and can also execute the description of the image processing device in the above corresponding embodiment, which will not be described again here. In addition, the description of the beneficial effects of using the same method will not be described again.
  • Embodiments of the present application also provide a computer-readable storage medium.
  • An image processing program is stored on the storage medium.
  • the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.
  • FIG 10 is a schematic diagram of the hardware structure of a controller according to an embodiment of the present application.
  • the controller 140 includes: a memory 1401 and a processor 1402.
  • the memory 1401 is used to store program instructions.
  • the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the above method embodiment. Its implementation principle and The beneficial effects are similar and will not be described again here.
  • the above-mentioned controller also includes a communication interface 1403, which can be connected to the processor 1402 through a bus 1404.
  • the processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.
  • FIG 11 is a schematic diagram of the hardware structure of a network node according to an embodiment of the present application.
  • the network node 150 includes: a memory 1501 and a processor 1502.
  • the memory 1501 is used to store program instructions.
  • the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the above method embodiment. Its implementation principle and The beneficial effects are similar and will not be described again here.
  • the above-mentioned network node also includes a communication interface 1503, which can be connected to the processor 1502 through a bus 1504.
  • the processor 1502 can control the communication interface 1503 to implement the receiving and transmitting functions of the network node 150 .
  • Embodiments of the present application also provide a computer program product.
  • the computer program product includes computer program code.
  • the computer program code When the computer program code is run on a computer, it causes the computer to execute the methods in the above various possible implementations.
  • Embodiments of the present application also provide a chip, which includes a memory and a processor.
  • the memory is used to store a computer program.
  • the processor is used to call and run the computer program from the memory, so that the device equipped with the chip executes the above various possible implementations. Methods.
  • the units in the equipment of the embodiments of this application can be merged, divided, and deleted according to actual needs.
  • the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation.
  • the technical solution of the present application can be embodied in the form of a software product in essence or that contributes to the existing technology.
  • the computer software product is stored in one of the above storage media (such as ROM/RAM, magnetic disk, optical disk), including several instructions to cause a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.
  • a computer program product includes one or more computer instructions.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g., computer instructions may be transmitted from a website, computer, server or data center via a wired link (e.g.
  • Coaxial cable, optical fiber, digital subscriber line) or wireless means to transmit to another website, computer, server or data center.
  • Computer-readable storage media can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or other integrated media that contains one or more available media. Available media may be magnetic media (eg, floppy disks, storage disks, tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种图像处理方法、智能终端及存储介质，该图像处理方法包括以下步骤：S1：获取或确定图像块的目标预测模式；S2：根据所述目标预测模式确定所述图像块的预测结果。通过本申请实施例，可以灵活地使用目标预测模式，保证预测结果的准确度。

Description

图像处理方法、智能终端及存储介质
本申请要求于2022年07月15日提交中国专利局、申请号为202210832482.1、申请名称为“图像处理方法、智能终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，具体涉及一种图像处理方法、智能终端及存储介质。
背景技术
随着视频编解码标准的不断更迭，视频编解码技术越来越成熟。在视频编码或解码中，对图像块进行预测是一项十分核心的技术。通常，在编码端，通过预测可以得到实际像素值与预测像素值的差值，该值为预测残差进行编码，以高效地进行传输；在解码端，解码端可以解码得到预测残差，并采用和编码端相同的预测方式得到相同的预测值，进而基于预测残差和预测值重建图像。
在构思及实现本申请过程中，发明人发现至少存在如下问题：一些预测模式的灵活度不高，预测处理方式受限。
前面的叙述在于提供一般的背景信息，并不一定构成现有技术。
申请内容
针对上述技术问题，本申请提供一种图像处理方法、智能终端及存储介质，可以灵活地使用目标预测模式，保证预测结果的准确性。
为解决上述技术问题，本申请提供一种图像处理方法，可应用于智能终端，包括以下步骤：
S1：获取或确定图像块的目标预测模式；
S2：根据所述目标预测模式确定所述图像块的预测结果。
可选地，所述图像块被划分为第一分区和第二分区，所述目标预测模式包括第一帧内预测模式、第一帧间预测模式、第二帧内预测模式、第二帧间预测模式中的至少一个。并且，所述第一分区采用第一帧内预测模式或者第一帧间预测模式，或者所述第二分区采用第二帧内预测模式或者第二帧间预测模式。
可选地，所述第一帧间预测模式为单向预测模式或者双向预测模式，或者，所述第二帧间预测模式为单向预测模式或者双向预测模式。
可选地，所述目标预测模式包括基于模板匹配的几何划分模式，所述图像块在所述基于模板匹配的几何划分模式下被划分为第一分区和第二分区。
可选地，包括以下至少一项：所述基于模板匹配的几何划分模式的类型包括：基于模板匹配的帧内预测和/或基于模板匹配的帧间预测；所述第一分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测；所述第二分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测。
在一个实施例中，所述目标预测模式包括基于模板匹配的几何划分模式，所述S2步骤，包括：通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板；根据所述相似模板确定所述图像块的预测结果。
在一个实施例中，所述通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板，包括：根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板。
可选地，所述预定搜索范围基于参考块对应的相似模板和/或参考块的预测候选块的位置确定。
在一个实施例中，所述方法还包括：根据所述参考块和所述图像块之间的位置关系、所述参考块对应的相似模板的位置，确定所述预定搜索范围；和/或，根据所述参考块和所述图像块之间的位置关系、所述参考块的预测候选块的位置，确定所述预定搜索范围。
在一个实施例中，所述根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板，包括：若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则从所述图像块所在的图像中确定当前模板对应的至少一个相似模板；和/或，若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板。
在一个实施例中，所述从所述图像块所在的图像中确定当前模板对应的至少一个相似模板，包括：按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板；根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
在一个实施例中，所述从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板，包括：从所述图像块所在的图像对应的第一参考图像中确定当前模板对应的第一相似模板，和/或，从所述图像块所在的图像对应的第二参考图像中确定当前模板对应的第二相似模板，将所述第一相似模板和/或所述第二相似模板确定为所述当前模板对应的至少一个相似模板；和/或，根据至少一个初始运动矢量从所述图像块所在的图像对应的参考图像中确定至少一个初始相似模板，在包括任一个所述初始相似模板的预定范围内搜索与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
可选地，包括以下至少一项：
所述按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板，包括：在对在所述图像块所在的图像中的重建图像区域进行搜索的过程中，当出现预设数量个满足匹配条件的目标区域，则将各个满足匹配条件的目标区域确定为与当前模板匹配的相似模板；和/或，当所述重建图像区域搜索完毕之后，将满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板；
所述根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板，包括：确定所匹配的相似模板与所述当前模板之间的模板代价，将满足预设模板条件的模板代价对应的所匹配的相似模板确定为所述当前模板对应的至少一个相似模板。
在一个实施例中，所述根据所述相似模板确定所述图像块的预测结果，包括：根据对应分区的至少一个相似模板确定该 分区的至少一个预测候选块；根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果。
在一个实施例中，所述方法还包括：获取所述图像块的初始当前模板；将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板。
在一个实施例中，S1步骤，包括以下至少一项：通过第一标志确定所述图像块的目标预测模式；将最小率失真代价对应的预测模式确定为所述图像块的目标预测模式。
可选地，若所述目标预测模式是基于模板匹配的几何划分模式，则在确定所述预测结果的过程中，利用第一标志指示所述图像块使用所述基于模板匹配的几何划分模式，以及利用至少一第二标志指示所述图像块的第一分区和/或第二分区各自使用的预测处理。
在一个实施例中，所述S2步骤，包括：根据所述目标预测模式，确定所述图像块的至少一个分区中的每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
在一个实施例中，若所述目标预测模式为基于模板匹配的几何划分模式，则所述S2步骤包括：确定所述图像块的初始当前模板，以用于通过所述初始当前模板确定所述图像块的预测结果；可选地，所述初始当前模板包括分区模板区域和非分区模板区域。
可选地，分区模板区域包括第一分区对应的第一当前模板以及第二分区对应的第二当前模板。
可选地，初始当前模板包括与所述图像块的上边界相邻的区域、与所述图像块的左边界相邻的区域、与所述图像块左上方的相邻区域中的至少一个区域；所述初始当前模板包括所述第一分区对应的当前模板和所述第二分区对应的当前模板。
可选地，所述第一分区对应的当前模板的横坐标范围与所述第二分区对应的当前模板的横坐标范围不同；或者，所述第一分区对应的当前模板的纵坐标范围与所述第二分区对应的当前模板的纵坐标范围不同；或者，所述第一分区对应的当前模板的长和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的长不相等；或者，所述第一分区对应的当前模板的宽和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的宽不相等。
可选地，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界，所述至少一边界位于所述图像块使用的分割线所在的直线上；或者，
所述初始当前模板包括所述图像块的第一分区对应的当前模板、所述图像块的第二分区对应的当前模板、以及非分区模板区域，其中所述图像块使用的分割线所在的直线经过所述非分区模板区域；或者所述非分区模板区域与所述图像块的第一分区对应的当前模板与所述图像块的第二分区对应的当前模板中的至少一个相邻。
可选地，所述当前模板的类型包括斜边模板和/或非斜边模板，所述至少一个相似模板的类型和所述当前模板的类型相同；当所述当前模板的类型为所述斜边模板时，所述斜边模板的斜边位置和斜率根据划分所述图像块的分割线设置；当所述当前模板的类型为所述非斜边模板时，所述非斜边模板的位置根据所述分割线所在直线经过所述初始当前模板的区域进行设置。
本申请还提供了一种装置，包括：
获取模块，用于获取或确定图像块的目标预测模式；
确定模块，用于根据所述目标预测模式确定所述图像块的预测结果。
本申请还提供一种智能终端，包括：存储器、处理器，其中，所述存储器上存储有图像处理程序，所述图像处理程序被所述处理器执行时实现如上任一所述图像处理方法的步骤。
本申请还提供一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如上任一所述图像处理方法的步骤。
如上，本申请的图像处理方法，可应用于智能终端，包括步骤：S1：获取或确定图像块的目标预测模式；S2：根据所述目标预测模式确定所述图像块的预测结果。图像块所使用的目标预测模式为任一种符合条件的预测模式，该预测模式可以是从至少一种预测模式中确定的，选择灵活性高；目标预测模式可以对应一种或多种预测处理方式，通过目标预测模式可对图像块进行灵活地预测，保证图像块的预测结果的准确性。可见，采用本方案可以实现灵活使用地目标预测模式，保证预测结果的准确性的功能，解决预测模式不灵活，预测处理方式受限的问题。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。为了更清楚地说明本申请实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为实现本申请各个实施例的一种移动终端的硬件结构示意图；
图2为本申请实施例提供的一种通信网络系统架构图；
图3a是根据第一实施例示出的图像处理方法的流程示意图；
图3b是根据第一实施例示出的一种分割线划分图像块的效果示意图；
图4a是根据第二实施例示出的图像处理方法的流程示意图；
图4b是根据第二实施例示出的一种角度划分的示意图；
图4c是根据第二实施例示出的一种在角度φi下对应的多种偏移的示意图；
图4d是根据第二实施例示出的一种划分模式的示意图；
图5a是根据第二实施例示出的一种初始当前模板的示意图；
图5b是根据第二实施例示出的一种划分初始当前模板的效果示意图；
图5c是根据第二实施例示出的另一种划分初始当前模板的效果示意图；
图6是根据第三实施例示出的一种图像处理方法的流程示意图；
图7a是根据第三实施例示出的图像块所在图像中的重建图像区域；
图7b是根据第三实施例示出的一种基于模板匹配的帧内预测的示意图；
图7c是根据第三实施例示出的一种基于模板匹配的帧间预测的示意图；
图7d是根据第三实施例示出的一种运动矢量候选列表的相邻块位置示意图；
图7e是根据第三实施例示出的一种运动矢量候选列表的示意图；
图7f是根据第三实施例示出的另一种基于模板匹配的帧间预测的示意图；
图7g是根据第三实施例示出的一种各个分区采用基于模板匹配的帧间预测的示意图；
图7h是根据第三实施例示出的采用的基于模板匹配的几何划分模式的类型不同时的预测示意图；
图8是根据第四实施例示出的一种图像处理方法的流程示意图；
图9是本申请实施例示出的一种图像处理装置的结构示意图；
图10是本申请实施例示出的一种控制器的硬件结构示意图；
图11是本申请实施例示出的一种网络节点的硬件结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。通过上述附图，已示出本申请明确的实施例，后文中将有更详细的描述。这些附图和文字描述并不是为了通过任何方式限制本申请构思的范围，而是通过参考特定实施例为本领域技术人员说明本申请的概念。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素，此外，本申请不同实施例中具有同样命名的部件、特征、要素可能具有相同含义，也可能具有不同含义，其具体含义需以其在该具体实施例中的解释或者进一步结合该具体实施例中上下文进行确定。
应当理解，尽管在本文可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本文范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语"如果"可以被解释成为"在……时"或"当……时"或"响应于确定"。再者，如同在本文中所使用的，单数形式“一”、“一个”和“该”旨在也包括复数形式，除非上下文中有相反的指示。应当进一步理解，术语“包含”、“包括”表明存在的特征、步骤、操作、元件、组件、项目、种类、和/或组，但不排除一个或多个其他特征、步骤、操作、元件、组件、项目、种类、和/或组的存在、出现或添加。本申请使用的术语“或”、“和/或”、“包括以下至少一个”等可被解释为包括性的，或意味着任一个或任何组合。例如，“包括以下至少一个：A、B、C”意味着“以下任一个：A；B；C；A和B；A和C；B和C；A和B和C”，再如，“A、B或C”或者“A、B和/或C”意味着“以下任一个：A；B；C；A和B；A和C；B和C；A和B和C”。仅当元件、功能、步骤或操作的组合在某些方式下内在地互相排斥时，才会出现该定义的例外。
应该理解的是，虽然本申请实施例中的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，其可以以其他的顺序执行。而且，图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，其执行顺序也不必然是依次进行，而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
取决于语境，如在此所使用的词语“如果”、“若”可以被解释成为“在……时”或“当……时”或“响应于确定”或“响应于检测”。类似地，取决于语境，短语“如果确定”或“如果检测(陈述的条件或事件)”可以被解释成为“当确定时”或“响应于确定”或“当检测(陈述的条件或事件)时”或“响应于检测(陈述的条件或事件)”。
需要说明的是，在本文中，采用了诸如S301、S302等步骤代号，其目的是为了更清楚简要地表述相应内容，不构成顺序上的实质性限制，本领域技术人员在具体实施时，可能会先执行S302后执行S301等，但这些均应在本申请的保护范围之内。
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在后续的描述中，使用用于表示元件的诸如“模块”、“部件”或者“单元”的后缀仅为了有利于本申请的说明，其本身没有特定的意义。因此，“模块”、“部件”或者“单元”可以混合地使用。
智能终端可以以各种形式来实施。例如，本申请中描述的智能终端可以包括诸如手机、平板电脑、笔记本电脑、掌上电脑、个人数字助理(Personal Digital Assistant，PDA)、便捷式媒体播放器(Portable Media Player，PMP)、导航装置、可穿戴设备、智能手环、计步器等智能终端，以及诸如数字TV、台式计算机等固定终端。
后续描述中将以移动终端为例进行说明，本领域技术人员将理解的是，除了特别用于移动目的的元件之外，根据本申请的实施方式的构造也能够应用于固定类型的终端。
请参阅图1，其为实现本申请各个实施例的一种移动终端的硬件结构示意图，该移动终端100可以包括：RF(Radio Frequency，射频)单元101、WiFi模块102、音频输出单元103、A/V(音频/视频)输入单元104、传感器105、显示单元106、用户输入单元107、接口单元108、存储器109、处理器110、以及电源111等部件。本领域技术人员可以理解，图1中示出的移动终端结构并不构成对移动终端的限定，移动终端可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图1对移动终端的各个部件进行具体的介绍：
射频单元101可用于收发信息或通话过程中，信号的接收和发送，具体的，将基站的下行信息接收后，给处理器110处理；另外，将上行的数据发送给基站。通常，射频单元101包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等。此外，射频单元101还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议，包括但不限于GSM(Global System of Mobile communication，全球移动通讯系统)、GPRS(General Packet  Radio Service，通用分组无线服务)、CDMA2000(Code Division Multiple Access 2000，码分多址2000)、WCDMA(Wideband Code Division Multiple Access,宽带码分多址)、TD-SCDMA(Time Division-Synchronous Code Division Multiple Access，时分同步码分多址)、FDD-LTE(Frequency Division Duplexing-Long Term Evolution，频分双工长期演进)、TDD-LTE(Time Division Duplexing-Long Term Evolution，分时双工长期演进)和5G等。
WiFi属于短距离无线传输技术，移动终端通过WiFi模块102可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。虽然图1示出了WiFi模块102，但是可以理解的是，其并不属于移动终端的必须构成，完全可以根据需要在不改变发明的本质的范围内而省略。
音频输出单元103可以在移动终端100处于呼叫信号接收模式、通话模式、记录模式、语音识别模式、广播接收模式等等模式下时，将射频单元101或WiFi模块102接收的或者在存储器109中存储的音频数据转换成音频信号并且输出为声音。而且，音频输出单元103还可以提供与移动终端100执行的特定功能相关的音频输出(例如，呼叫信号接收声音、消息接收声音等等)。音频输出单元103可以包括扬声器、蜂鸣器等等。
A/V输入单元104用于接收音频或视频信号。A/V输入单元104可以包括图形处理器(Graphics Processing Unit，GPU)1041和麦克风1042，图形处理器1041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。处理后的图像帧可以显示在显示单元106上。经图形处理器1041处理后的图像帧可以存储在存储器109(或其它存储介质)中或者经由射频单元101或WiFi模块102进行发送。麦克风1042可以在电话通话模式、记录模式、语音识别模式等等运行模式中经由麦克风1042接收声音(音频数据)，并且能够将这样的声音处理为音频数据。处理后的音频(语音)数据可以在电话通话模式的情况下转换为可经由射频单元101发送到移动通信基站的格式输出。麦克风1042可以实施各种类型的噪声消除(或抑制)算法以消除(或抑制)在接收和发送音频信号的过程中产生的噪声或者干扰。
移动终端100还包括至少一种传感器105，比如光传感器、运动传感器以及其他传感器。可选地，光传感器包括环境光传感器及接近传感器，可选地，环境光传感器可根据环境光线的明暗来调节显示面板1061的亮度，接近传感器可在移动终端100移动到耳边时，关闭显示面板1061和/或背光。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；至于手机还可配置的指纹传感器、压力传感器、虹膜传感器、分子传感器、陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
显示单元106用于显示由用户输入的信息或提供给用户的信息。显示单元106可包括显示面板1061，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode，OLED)等形式来配置显示面板1061。
用户输入单元107可用于接收输入的数字或字符信息，以及产生与移动终端的用户设置以及功能控制有关的键信号输入。可选地，用户输入单元107可包括触控面板1071以及其他输入设备1072。触控面板1071，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板1071上或在触控面板1071附近的操作)，并根据预先设定的程式驱动相应的连接装置。触控面板1071可包括触摸检测装置和触摸控制器两个部分。可选地，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器110，并能接收处理器110发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板1071。除了触控面板1071，用户输入单元107还可以包括其他输入设备1072。可选地，其他输入设备1072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种，具体此处不做限定。
可选地，触控面板1071可覆盖显示面板1061，当触控面板1071检测到在其上或附近的触摸操作后，传送给处理器110以确定触摸事件的类型，随后处理器110根据触摸事件的类型在显示面板1061上提供相应的视觉输出。虽然在图1中，触控面板1071与显示面板1061是作为两个独立的部件来实现移动终端的输入和输出功能，但是在某些实施例中，可以将触控面板1071与显示面板1061集成而实现移动终端的输入和输出功能，具体此处不做限定。
接口单元108用作至少一个外部装置与移动终端100连接可以通过的接口。例如，外部装置可以包括有线或无线头戴式耳机端口、外部电源(或电池充电器)端口、有线或无线数据端口、存储卡端口、用于连接具有识别模块的装置的端口、音频输入/输出(I/O)端口、视频I/O端口、耳机端口等等。接口单元108可以用于接收来自外部装置的输入(例如，数据信息、电力等等)并且将接收到的输入传输到移动终端100内的一个或多个元件或者可以用于在移动终端100和外部装置之间传输数据。
存储器109可用于存储软件程序以及各种数据。存储器109可主要包括存储程序区和存储数据区，可选地，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器109可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
处理器110是移动终端的控制中心，利用各种接口和线路连接整个移动终端的各个部分，通过运行或执行存储在存储器109内的软件程序和/或模块，以及调用存储在存储器109内的数据，执行移动终端的各种功能和处理数据，从而对移动终端进行整体监控。处理器110可包括一个或多个处理单元；优选的，处理器110可集成应用处理器和调制解调处理器，可选地，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器110中。
移动终端100还可以包括给各个部件供电的电源111(比如电池)，优选的，电源111可以通过电源管理系统与处理器110逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。
尽管图1未示出，移动终端100还可以包括蓝牙模块等，在此不再赘述。
为了便于理解本申请实施例，下面对本申请的移动终端所基于的通信网络系统进行描述。
请参阅图2，图2为本申请实施例提供的一种通信网络系统架构图，该通信网络系统为通用移动通信技术的LTE系统，该LTE系统包括依次通讯连接的UE(User Equipment，用户设备)201，E-UTRAN(Evolved UMTS Terrestrial Radio Access Network，演进式UMTS陆地无线接入网)202，EPC(Evolved Packet Core，演进式分组核心网)203和运营商的IP业务204。
可选地，UE201可以是上述移动终端100，此处不再赘述。
E-UTRAN202包括eNodeB2021和其它eNodeB2022等。可选地，eNodeB2021可以通过回程(backhaul)(例如X2接口)与其它eNodeB2022连接，eNodeB2021连接到EPC203，eNodeB2021可以提供UE201到EPC203的接入。
EPC203可以包括MME(Mobility Management Entity，移动性管理实体)2031，HSS(Home Subscriber Server，归属用户服务器)2032，其它MME2033，SGW(Serving Gate Way，服务网关)2034，PGW(PDN Gate Way，分组数据网络网关)2035和PCRF(Policy and Charging Rules Function，政策和资费功能实体)2036等。可选地，MME2031是处理UE201和EPC203之间信令的控制节点，提供承载和连接管理。HSS2032用于提供一些寄存器来管理诸如归属位置寄存器(图中未示)之类的功能，并且保存有一些有关服务特征、数据速率等用户专用的信息。所有用户数据都可以通过SGW2034进行发送，PGW2035可以提供UE201的IP地址分配以及其它功能，PCRF2036是业务数据流和IP承载资源的策略与计费控制策略决策点，它为策略与计费执行功能单元(图中未示)选择及提供可用的策略和计费控制决策。
IP业务204可以包括因特网、内联网、IMS(IP Multimedia Subsystem，IP多媒体子系统)或其它IP业务等。
虽然上述以LTE系统为例进行了介绍，但本领域技术人员应当知晓，本申请不仅仅适用于LTE系统，也可以适用于其他无线通信系统，例如GSM、CDMA2000、WCDMA、TD-SCDMA以及未来新的网络系统(如5G)等，此处不做限定。
基于上述移动终端硬件结构以及通信网络系统，提出本申请各个实施例。
第一实施例
请参阅图3a，图3a是根据第一实施例示出的一种图像处理方法的流程示意图，该实施例中的执行主体可以是一个计算机设备或者是多个计算机设备构成的集群，该计算机设备可以是智能终端(如前述移动终端100)，也可以是服务器，此处，以本实施例中的执行主体为智能终端为例进行说明。
S301，获取或确定图像块的目标预测模式。
步骤S301中的图像块是指输入视频图像(即视频帧)中当前正在编码的图像块，也可称为当前块，或者当前图像块，或者当前编码块。图像块可以是输入视频图像中的一个编码树单元(Coding Tree Units，CTU)，或者是一个编码单元(Coding Unit，CU)，或者是变换单元(Transform Unit，TU)等等。对此不做限制。图像块可以是方形块(即图像块的尺寸为正方形)，也可以非方形块。非方形块可以是矩形尺寸的图像块，包括水平块(宽度大于高度)和/或垂直块(高度大于宽度)，举例来说，当图像块为CU时，CU可以是方形块或者是非方形块。在此不做限制。本申请中，在不做特别说明之下，图像块和当前块是等同的。
在视频编码或解码的过程中，通常可以使用一种预测模式来进行指示具体的预测处理方式。广义上看，对图像块的预测处理一般包括帧内预测以及帧间预测，但是在此基础上，还可以通过多种变形组合构成其他新的预测模式，以应对不同的视频编解码场景。举例来说，为了实现更加准确的预测，针对图像中运动物体的边界部分，采用几何划分模式(Geometric Partition Mode，简称GPM模式)，该几何划分模式通过使用贴合物体边界的分割线将图像块划分为不同分区来进行帧间预测。
目标预测模式可以是图像块所支持的至少一种预测模式中的任一种预测模式。该预测模式用于对图像块进行预测以得到预测结果。在一实现方式中，目标预测模式包括帧内预测模式或者帧间预测模式中的至少一个。帧内预测模式或者帧间预测模式为单向预测模式或者双向预测模式。单向预测模式对应于一个运动矢量，双向预测模式对应于两个运动矢量。在一实现方式中，图像块被划分为第一分区和第二分区，第一分区采用第一帧内预测模式或者第一帧间预测模式，或者第二分区采用第二帧内预测模式或者第二帧间预测模式。目标预测模式包括第一帧内预测模式、第一帧间预测模式、第二帧内预测模式、第二帧间预测模式中的至少一个。在一种实现方式中，目标预测模式包括基于模板匹配的几何划分模式。基于模板匹配的几何划分模式是一种新型的几何划分模式，也可称为具有模板匹配(Template Matching，TM)的GPM模式。其不仅具备几何划分模式的功能，即通过分割线将图像块划分为不同图像区域，还可以结合模板进行预测。模板匹配是指通过使用模板确定的预测候选块来对图像块进行预测，模板是指与图像块关联的区域，例如与图像块左边界相邻的像素点所在的已编码区域。基于模板匹配的几何划分模式将模板匹配应用于GPM模式，可以对GPM模式下的图像块的不同图像区域进行相应预测处理，从而得到整个图像块的预测结果。可选地，帧内预测模式可以为基于模板匹配的帧内预测，帧间预测模式可以为基于模板匹配的帧间预测。
可选地，图像块在所述基于模板匹配的几何划分模式下被划分为第一分区和第二分区。由于基于模板匹配的几何划分模式本质上也是一种几何划分模式，依据几何划分模式的原理，图像块使用的预测模式为该基于模板匹配的几何划分模式时，可以由GPM模式下的分割线划分图像块以得到不同图像区域，此处图像区域包括第一分区和第二分区。第一分区和第二分区是图像块中的矩形或者是非矩形区域，且第一分区和第二分区是相对而言的。第一分区和第二分区可以是关于图像块的矩形、三角形、或梯形区域。举例来说，如图3b中的(1)所示，使用水平方向的分割线划分图像块，在水平方向的分割线上方的图像区域可以称为第一分区，在水平方向的分割线下方的图像区域可以称为第二分区。反之，也可以将在水平方向的分割线下方的图像区域称为第一分区，在水平方向的分割线上方的图像区域称为第二分区，且两个分区都是矩形区域。如图3b中的(2)所示，分割线划分的图像块得到的两个分区中，一个为三角形区域，另一个为非矩形区域。
基于模板匹配的几何划分模式可以包括至少一种类型，每种类型都可指示在几何划分模式下利用模板匹配的规则进行预测处理的逻辑。对于图像块至少一个分区(如第一分区和/或第二分区)中的各个分区，每个分区均可采用基于模板匹配的几何划分模式，但具体的类型可自由选择，预测灵活性高。可选地，所述第一分区和所述第二分区采用的基于模板匹配的几何划分模式的类型相同或者不同。
若图像块中各个分区采用的基于模板匹配的几何划分模式的类型相同，则各个分区使用相同预测处理的逻辑确定图像块的预测结果；若图像块中各个分区采用的基于模板匹配的几何划分模式的类型不同，则各个分区使用不同的预测处理的逻辑 确定图像块的预测结果。可见，对于基于模板匹配的几何划分模式提供至少一种类型，并且不同分区可使用的类型支持灵活地进行组合，进而可增加具有TM的GPM模式预测的灵活性。
可选地，基于模板匹配的几何划分模式的类型包括：基于模板匹配的帧内预测和/或基于模板匹配的帧间预测。基于模板匹配的帧内预测是利用帧内图像块的相关性来进行预测的方式，基于模板匹配的帧间预测是利用帧间图像的相关性来进行预测的方式。基于模板匹配的帧内预测可在几何划分模式下，在当前编码的图像块所在的已编码图像中查找与当前编码的图像块的模板所匹配的模板进行预测；基于模板匹配的帧间预测可在几何划分模式下，在当前编码的图像块所对应的参考帧中查找与当前编码的图像块的模板所匹配的模板进行预测，上述图像块的模板可以是图像块中任一分区对应的模板。基于模板匹配的帧间预测具体是基于模板匹配的帧间预测的几何划分模式，基于模板匹配的帧内预测具体是基于模板匹配的帧内预测的几何划分模式。通过提供这两种类型，基于模板匹配的GPM模式不仅支持帧间预测，还支持帧内预测，同一预测模式对应的预测处理方式丰富多样，灵活度高。
可选地，对于图像块中由基于模板匹配的几何划分模式所划分的各个分区，具体可以采用以上任一种基于模板匹配的几何划分模式的类型，即：所述第一分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测；所述第二分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测。第一分区和第二分区采用的基于模板匹配的几何划分模式的类型相同，包括：①第一分区和第二分区均采用基于模板匹配的帧内预测，②第一分区和第二分区均采用基于模板匹配的帧间预测。第一分区和第二分区采用的基于模板匹配的几何划分模式的类型不同，包括：①第一分区采用基于模板匹配的帧间预测，第二分区采用基于模板匹配的帧内预测；②第一分区采用基于模板匹配的帧内预测，第二分区采用基于模板匹配的帧间预测。对于整个图像块来说，当采用的基于模板匹配的几何划分模式的类型不同时，基于模板匹配的GPM模式具体是一种基于模板匹配的帧内预测和帧间预测相结合的GPM模式，当采用的基于模板匹配的几何划分模式的类型相同时，基于模板匹配的GPM模式具体是一种基于模板匹配的帧内预测的GPM模式，或者是一种基于模板匹配的帧间预测的GPM模式。可见，不同分区支持至少一种基于模板匹配的几何划分模式的类型，同一图像块在相同预测模式下可使用至少一种预测处理方式(包括帧间预测和/或帧内预测)进行预测，这样，图像块对基于模板匹配的几何划分模式的使用可以更加灵活和多样，从而增加具有模板匹配的GPM模式预测的灵活性。
对于图像块的目标预测模式的确定方式或获取方式可以有多种，例如根据各个预测模式应用于图像块的率失真代价确定，又例如，基于与当前块相邻的图像块所使用的预测模式确定等等。对于目标预测模式的获取或确定方式，可参见下述实施例，在此先不做详述。
S302，根据所述目标预测模式确定所述图像块的预测结果。
目标预测模式可指示对图像块进行预测处理的方式，例如，目标预测模式为基于模板匹配的几何划分模式，对于图像块的预测处理包括对图像块的分割处理以及模板匹配处理。基于目标预测模式对图像块进行预测可得到预测结果。举例来说，目标预测模式为基于模板匹配的几何划分模式，具体为基于模板匹配的帧间预测的几何划分模式，此预测模式下，可从图像块所在图像所对应的参考帧中查找与当前编码的图像块的模板所匹配的模板，基于所匹配的模板确定预测候选块，进而基于预测候选块确定出预测结果。
可选地，可根据目标预测模式和预设策略确定图像块的预测结果。预设策略用于指示在目标预测模式下对图像块的处理策略，该预设策略可与目标预测模式相匹配。例如，目标预测模式为基于模板匹配的几何划分模式，该预设策略可以是指示模板匹配的策略，具体用于查找出与图像块的模板所匹配的模板。
本申请实施例所提供的图像处理方法，可确定或获取图像块的目标预测模式，目标预测模式可从至少一种预测模式中选择，选择范围广且灵活性高。由于目标预测模式可指示对图像块进行预测处理的至少一种方式，通过目标预测模式可灵活地使用相应的预测处理方式对当前编码的图像块进行预测，保证预测结果的准确性。在此基础上，当目标预测模式包括基于模板匹配的几何划分模式时，由于基于模板匹配的几何划分模式的类型不仅包括基于模板匹配的帧间预测，还包括基于模板匹配的帧内预测，而图像块中不同分区可以采用相同或不同类型的基于模板匹配的几何划分模式，图像块可以组合至少一种类型的基于模板匹配的几何划分模式，使用不同的预测处理方式进行预测，从而使得图像块使用基于模板匹配的几何划分模式更加灵活。
第二实施例
请参阅图4a，图4a是根据第二实施例示出的一种图像处理方法的流程示意图，该实施例中的执行主体可以是一个计算机设备或者是多个计算机设备构成的集群，该计算机设备可以是智能终端(如前述移动终端100)，也可以是服务器，此处，以本实施例中的执行主体为智能终端为例进行说明。
S401，获取或确定图像块的目标预测模式。
本申请实施例可以应用于编码器和/或解码器，对于编码端和解码端，S401的处理方式可以相同或不同。在一个实施例中，S401包括：通过第一标志确定所述图像块的目标预测模式。第一标志可以是传输于比特流中的标志，也可以是传输于CTU或者CU中的标志。可选地，所述目标预测模式包括基于模板匹配的几何划分模式，所述第一标志用于指示所述图像块是否使用基于模板匹配的几何划分模式(或者说，对于所述图像块是否将模板应用于几何划分模式)。举例来说，第一标志为1，表示图像块使用基于模板匹配的几何划分模式，和/或，第二标志为0，表示图像块不使用基于模板匹配的几何划分模式，而是使用其他预测模式。此方式可应用于解码端或编码端。
在一种实现方式中，若所述第一标志指示所述图像块使用基于目标匹配的划分模式，则利用至少一第二标志指示所述图 像块的第一分区和第二分区是否使用基于模板匹配的帧内预测和基于模板匹配的帧间预测中的至少一个。第二标志可以是一编码单元级的标志，第二标志用于指示图像块各分区使用基于模板匹配的帧内预测以及基于模板匹配的帧间预测中的至少一个。例如第二标志指示图像块中第一分区使用基于模板匹配的帧内预测，以及第二分区使用基于模板匹配的帧间预测。可以使用一个或多个第二标志来指示以上内容，在一种方式中，第二标志以二进制表示，具体使用两位“1”和“0”的排列组合来表示，例如00、01，第一位表示图像块中一分区(例如第一分区)使用基于模板匹配的帧内预测或基于模板匹配的帧间预测，第二位表示图像块另一分区(例如第二分区))使用基于模板匹配的帧内预测或基于模板匹配的帧间预测，1代表使用基于模板匹配的帧内预测，和/或，0代表使用基于模板匹配的帧间预测。例如第二标志为00表示第一分区和第二分区均使用基于模板匹配的帧间预测，此方式下可通过一个第二标志指示图像块中各个分区采用的基于模板匹配的几何划分模式的类型。在另一种方式中，第二标志也可以通过其他字符表示，例如数字和字母的组合，数字1代表第一分区的基于模板匹配的几何划分模式的类型，和/或，0代表第二分区的基于模板匹配的几何划分模式的类型，字母a表示使用基于模板匹配的帧间预测，字母b表示使用基于模板匹配的帧内预测。例如1-a可用于表示第一分区采用基于模板匹配的帧间预测，此方式下可通过多个第二标志指示图像块中各个分区采用的基于模板匹配的几何划分模式的类型。可以理解的是，第二标志的表示并不局限于上述所介绍的方式。可见，通过第一标志和第二标志结合的指示，可以快速便捷地确定出图像块所使用的预测模式，十分简单方便。
在一个实施例中，若所述目标预测模式是基于模板匹配的几何划分模式，则在确定所述预测结果的过程中，利用第一标志指示所述图像块使用所述基于模板匹配的几何划分模式，以及利用至少一第二标志指示所述图像块的第一分区和/或第二分区各自使用的预测处理。
第一分区和/或第二分区使用的预测处理包括：基于模板匹配的帧内预测和基于模板匹配的帧间预测。至少一第二标志指示所述图像块的第一分区和/或第二分区是否使用基于模板匹配的帧内预测和基于模板匹配的帧间预测中的至少一个。此处第一标志和第二标志可以同前述介绍的内容，在此不做赘述。在确定预测结果的过程中，通过各种标志的指示可以直接快速地确定出基于模板匹配的GPM模式以及此种预测模式下对应分区的预测处理是何种。应用于编码端或解码端时是一种十分高效的方式，可以快速地确定出预测结果，提高编码或解码效率。
在另一个实施例中，S401包括：将最小率失真代价对应的预测模式确定为所述图像块的目标预测模式。此方式可应用于编码端，首先可确定至少一种预测模式中每种预测模式应用于图像块的率失真代价，再通过各种预测模式的率失真代价的比较，确定出最小率失真代价对应的预测模式，并将其作为图像块的目标预测模式。
编码器可确定当前块的颜色分量(包括，亮度分量和/或色度分量)；基于当前块的参数，利用至少一种预测模式(如帧内预测和/或帧间预测)分别对颜色分量进行预测编码，计算每一种预测模式对应的率失真代价，以从至少一种预测模式分别对应的率失真代价中确定最小率失真代价。最后，将最小率失真代价对应的预测模式确定为当前块的目标预测模式。当最小率失真代价对应的预测模式为基于模板匹配的GPM模式时，则将基于模板匹配的GPM模式确定为当前块的目标预测模式。需要说明的是，在计算基于模板匹配的GPM模式的率失真代价时，可确定每种GMP模式的分割线类型下的当前块各分区对应的模板。在此条件下，进一步分别在各个分区采用基于模板匹配的帧内预测和基于模板匹配的帧间预测的情况下，可确定当前块的预测结果对应的率失真代价，以用于同其他预测模式下当前块的预测结果对应的率失真代价进行比较，从而确定出当前块各个分区中基于模板匹配的几何划分模式的类型。
在一种实现方式中，由于基于模板匹配的GPM模式是一种将模板应用于GPM模式的预测模式，因此，可以先确定出GPM模式的分割线类型。具体包括如下内容：通过遍历GPM模式对应的64种划分模型，确定出率失真代价最小的划分模式。之后，将出率失真代价最小的划分模式作为当前块的目标划分模式。示例性地，64种划分模型如表1所示。表1为GPM分割索引gpm_partition_idx、角度索引angleIdx和距离索引distanceIdx的映射表。最后，确定出率失真代价最小的划分模式包括确定该划分模式(即，当前块的目标划分模式)对应的目标GPM分割索引gpm_partition_idxT、目标角度索引angleIdxT和目标距离索引distanceIdxT。
表1、64种划分模型
需要说明的是，角度索引angleIdx取不同值时，对应于如图4b所示的不同角度φi，此处i为1～24。距离索引distanceIdx取不同值时，对应于如图4c中的ρj，此处j为0～3。如表1中的角度索引angleIdx以及距离索引distanceIdx的不同取值的组合分别构成不同的划分方式。具体可如图4d所示。
在一种可行的实现方式中，可以利用一查找表来表示GPM分割索引gpm_partition_idx与角度索引angleIdx，距离索引distanceIdx之间的关系。例如，如下表2所示。
表2、查找表
通过查找表可基于所记录的某一种索引(例如角度索引)快捷地查找到对应的数据，从而高效地确定出GPM模式的分割线类型。
在一个实施例中，角度索引angleIdx对应于角度的正弦值，距离索引distanceIdx对应于ρj。在一种实施方式中，可以设置关于angleIdx与cos(φ)的映射表，如表3所示。在另一实施方式中，还可以设置关于angleIdx的中间变量与cos(φ)的映射表。
表3、角度参数映射表
通过角度映射表的设置，可以着重关注角度参数的变化，通过角度参数的更新来更新角度索引，进而更新查找表中的距离索引和/或角度索引。
在一个实施例中，编码器可将目标预测模式对应的预测模式参数封装至比特流中进行传输。例如目标预测模式为基于模板匹配的GPM模式，可将基于模板匹配的GPM模式对应的基于模板匹配的GPM参数二值化后，打包至比特流中进行传输。此处基于模板匹配的GPM参数是一种预测模式参数。
当所述目标预测模式为基于模板匹配的几何划分模式时，所述预测模式参数包括以下至少一种：用于指示是否使用基于模板匹配的几何划分模式的信息(即，预测模式信息)、用于指示基于模板匹配的几何划分模式的类型的预测模式类型信息以及用于指示所述划分模式的分割线类型的划分模式类型信息。预测模式参数为基于模板匹配的GPM参数，通过该预测模式参数所包括的信息，基于模板匹配的GPM参数可用于指示图像块是否采用基于模板匹配的GPM模式、GPM模式的分割线类型、基于模板匹配的GPM模式类型中的至少一个。
在解码端，可以通过解析接收到的比特流获取预测模式参数。在一个实施例中，若预测模式参数包括指示当前块(即当前编码的图像块)是否采用基于模板匹配的GPM模式的参数，具体是指示当前块采用基于模板匹配的GPM模式的参数，则确定当前块的GPM参数。当前块的GPM参数包括指示GPM模式的分割线类型的信息。在一实施例中，GPM模式的分割线类型可以由关于GPM参数的GPM分割索引gpm_partition_idx来表示。在此实施例中，可以通过GPM分割索引gpm_partition_idx确定对应的角度索引angleIdx，距离索引distanceIdx(也称为步长索引)。可选地，GPM参数还可以包括其他参数。例如，GPM参数还可以包括第一分区使用的预测类型的标记或索引(即，第一分区对应的基于模板匹配的GPM模式的类型)，第二分区使用的预测类型的标记或索引(即，第二分区对应的基于模板匹配的GPM模式的类型)。可选地，GPM参数还可以包括GPM模式下的运动矢量索引。例如，gpm_idx0[x0][y0]和gpm_idx1[x0][y0]。gpm_idx0[x0][y0]，表示第一运动矢量在合并候选列表中的位置，gpm_idx1[x0][y0]，表示第二运动矢量在合并候选列表中的位置。第一运动矢量和第二运动矢量可以是第一分区的运动矢量，或者第一运动矢量和第二运动矢量可以是第二分区的运动矢量。举例来说，当第一分区使用的预测类型的标记或索引指示第一分区采用基于模板匹配的帧间预测模式，第二分区使用的预测类型的标记或索引指示第二分区采用基于模板匹配的帧内预测模式时，第一运动矢量和第二运动矢量是第一分区的运动矢量。当第一分区使用的预测类型的标记或索引指示第一分区采用基于模板匹配的帧内预测模式，第二分区使用的预测类型的标记或索引指示第二分区采用基于模板匹配的帧间预测模式时，第一运动矢量和第二运动矢量是第二分区的运动矢量。可见，解码端通过解析预测模式参数可以直接获取和编码端相同的预测处理方式，从而实现快捷地解码处理。
在解码端基于模板匹配的GPM模式的预测方式可以包括以下步骤：(1)自比特流中获取或确定预测模式信息，该预测模式信息用于指示图像块是否采用基于模板匹配的GPM模式。(2)若确定采用基于模板匹配的GPM模式，则进一步确定GPM模式的分割线类型、当前块的模板、以及基于模板匹配的GPM预测模式类型。在一种实现方式中，确定基于模板匹配的GPM模式类型包括确定是否使用基于模板匹配的帧内预测模式，和/或确定是否使用基于模板匹配的帧间预测模式，可以自比特流中获取或确定基于模板匹配的GPM预测模式类型信息，该基于模板匹配的GPM预测模式类型信息用于指示图像块是否使用基于模板匹配的帧内预测模式，和/或是否使用基于模板匹配的帧间预测模式；确定图像块的模板包括确定 模板的大小、形状、以及各模板对应的分区。模板的确定和分割线的类型相关联。例如，可以依据分割线的位置确定模板的位置，形状，以及对应的分区。之后可以按照S402和S403介绍的内容确定图像块的预测结果。
由于GPM模式的分割线类型包含至少一种，模板也有至少一种设计样式，具体的预测处理还包括帧内预测和帧间预测，通过组合这些内容可以构建基于模板匹配的GPM模式，对基于模板匹配的GPM模式可以划分出许多更加细致的类型，例如以64种分割线类型、2种预测模式为准可以组合出128种类型。由此，基于模板匹配的GPM模式极大地扩展了预测模式的种类，从而使得图像块中各分区使用的预测模式选择性更广，有助于增加基于模板匹配的几何划分模式的灵活性。
在一个实施例中，目标预测模式包括基于模板匹配的几何划分模式。在此种预测模式下，根据目标预测模式确定图像块的预测结果的实现方式可以包括以下S402和S403中所描述的内容。
在一个可行的实施例中，若所述目标预测模式为基于模板匹配的几何划分模式，根据目标预测模式确定图像块的预测结果的实现方式还可以是：确定所述图像块的初始当前模板，以用于通过所述初始当前模板确定所述图像块的预测结果。针对此方式下的具体实现内容，也可以采用S402～S403所描述方案实现。
S402，通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板。
图像块在使用基于模板匹配的几何划分模式时，可以基于所述模板匹配的几何划分模式包括的分割线类型所对应的分割线，对图像块进行划分得到多个图像区域，并将其作为图像块的各个分区。图像块中每个分区均具备相应的模板，在此称为当前模板，该当前模板包括与图像块对应分区相邻的已编码像素点所在的区域。当前模板的形状可以是矩形，也可以是其他形状，例如梯形。此处预设策略可用于指示其他模板与图像块的当前模板进行匹配的准则，基于该准则可查找出与图像块的当前模板所匹配的相似模板。
在一个可行的实施例中，执行S402之前，可确定图像块中各个分区对应的当前模板。图像块的各个分区包括第一分区和第二分区，在一种实现方式中，各个分区对应的当前模板的确定方式可以为：获取图像块的初始当前模板；基于分割线对所述初始当前模板进行划分，得到图像块的第一分区对应的当前模板以及图像块的第二分区对应的当前模板。
初始当前模板是对于当前编码的图像块而言的，可选地，初始当前模板包括与图像块的上边界相邻的区域、与图像块的左边界相邻的区域、与图像块左上方的相邻区域中的至少一个区域。在一种实现方式中，初始当前模板可以包括图像块的上边界相邻的区域、与图像块的左边界相邻的区域、与图像块左上方的相邻区域；在另一种实现方式中，初始当前模板可以包括与图像块的上边界相邻的区域、与图像块的左边界相邻的区域。示例性地，如图5a所示的初始当前模板的示意图，包括以上两种实现方式下的初始当前模板。可以理解的是，以上仅为示例内容，还可以包括其他区域的组合方式，例如初始当前模板包括与图像块的上边界相邻的区域以及与图像块左上方的相邻区域。对于初始当前模板的区域的其他组合方式，在此不一一列举出。
在一种实现方式中，基于分割线对所述初始当前模板进行划分，得到图像块的第一分区对应的当前模板以及图像块的第二分区对应的当前模板，可以包括以下内容：通过所述图像块使用的分割线所在的直线，将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和所述图像块的第二分区对应的当前模板。可选地，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界位于所述图像块使用的分割线所在的直线上。第一分区对应的当前模板和第二分区对应的当前模板的至少一边界可以是两个分区各自对应的当前模板的相邻边。
在一种方式下，按照GPM模式下的分割线对初始当前模板进行划分，可以得到分区模板区域，该分区模板区域包括各个分区对应的模板区域，即包括第一分区对应的当前模板(可称为第一当前模板)和第二分区对应的当前模板(可称为第二当前模板)。初始当前模板与各分区的当前模板之间的关系，可以为：初始当前模板包括第一分区对应的当前模板、第二分区对应的当前模板。此时，第一分区对应的当前模板和第二分区对应的当前模板是相邻的，并且相邻边与分割线所在的直线重合。
示例性地，请参见图5b所示的划分初始当前模板的效果示意图。如图5b中的(1)和(2)所示，在当前块中包括第一分区和第二分区，第一分区对应的当前模板为当前模板A，第二分区对应的当前模板为当前模板B，当前模板A和当前模板B相邻，且相邻边位于分割线所在的直线上。图5b中的(1)和(2)不同之处在于初始当前模板所包含的区域不同。与图5a中的示例(1)相比，图5b中的示例(2)中的第一分区的当前模板A和第二分区的当前模板B均不包含与图像块左上方的相邻区域。可选地，图5b中的示例(1)的初始当前模板包含与图像块的上边界相邻的区域、与图像块的左边界相邻的区域，以及图像块左上方的相邻区域，而图5b中的(2)的初始当前模板仅包含与图像块的上边界相邻的区域、与图像块的左边界相邻的区域。因此，对于不同的初始当前模板，第一分区的当前模板A和第二分区的当前模板B包括的区域不同。
此外，对于不同的GPM分割模式，第一分区的当前模板A和第二分区的当前模板B包括的区域也不同。如图5b中的示例(1)所示，第一分区的当前模板A包括与图像块的上边界相邻的区域、与图像块的左边界相邻的区域、图像块左上方的相邻区域，并且第二分区的当前模板B包括与图像块的上边界相邻的区域。需要说明的是，图5b中的示例(1)仅用于说明本发明的基本思想。第一分区的当前模板A和第二分区的当前模板B包括的区域不是固定不变的，而会根据不同的GPM分割模式(即，对应不同的分割线)，第一分区的当前模板A和第二分区的当前模板B包括的区域不同。例如，若GPM分割模式对应于分割线位于右上45度对角线上，则在图5b中的示例(1)中第一分区的当前模板A包括与图像块的左边界相邻的区域和一部分图像块左上方的相邻区域，并且第二分区的当前模板B包括与图像块的上边界相邻的区域和一部分图像块左上方的相邻区域，在图5b的示例(2)中第一分区的当前模板A仅包括与图像块的左边界相邻的区域，并且第二分区的当前模板B仅包括与图像块的上边界相邻的区域。在其他实施方式中，对于图5b的示例(1)，根据不同的GPM分割模 式，当前模块A和当前模板B可以包括至少一部分当前块正上方的相邻区域、至少一部分当前块左侧的相邻区域、和至少一部分当前块左上方的相邻区域中的至少一个或其组合；对于图5b的示例(2)，根据不同的GPM分割模式，当前模块A和当前模板B可以包括至少一部分当前块正上方的相邻区域、至少一部分当前块左侧的相邻区域中的至少一个或其组合。在一实施例中，当前模块A和当前模板B中间的一个包括至少一部分当前块正上方的相邻区域和至少一部分当前块左侧的相邻区域，当前模块A和当前模板B中间的另一个仅包括至少一部分当前块正上方的相邻区域或至少一部分当前块左侧的相邻区域。可选地，当前模块A和当前模板B中间的另一个仅包括一部分当前块正上方的相邻区域或一部分当前块左侧的相邻区域，或者当前模块A和当前模板B中间的另一个仅包括整个当前块正上方的相邻区域或整个当前块左侧的相邻区域。
在另一种方式下，按照GPM模式下的分割线对初始当前模板进行划分，可以得到分区模板区域和非分区模板区域。非分区模板区域是不作为任一个分区的模板区域，属于不可用区域，图像块所使用的分割线所在的直线经过该非分区模板区域，并且非分区模板区域和分区模板区域是相邻的。初始当前模板与各分区的当前模板之间的关系可以为：初始当前模板包括所述图像块的第一分区对应的当前模板、所述图像块的第二分区对应的当前模板、以及非分区模板区域。可选地，在初始当前模板中部分模板区域并不能作为分区对应的当前模板。此时，第一分区的当前模板和第二分区对应的当前模板是不相邻的。
示例性地，请参见图5c所示另一种划分初始当前模板的效果示意图，如图5b中的(1)和(2)所示，在当前块中包括第一分区和第二分区，第一分区对应的当前模板为当前模板A，第二分区对应的当前模板为当前模板B，当前模板A和当前模板B不相邻，当前模板A和当前模板B的坐标范围(例如横坐标范围)不重合，当前模板A和当前模板B中间有间隔区域。由于分割线所在的直线经过该间隔区域，因此该间隔区域中可能包含物体边缘，为不可用区域。当前模板A和当前模板B均与不可用区域相邻，该不可用区域即非分区模板。同图5b，图5c中示例(1)和示例(2)中的初始当前模板所包含的区域不同。即，图5c中示例(1)包含左上方相邻区域，而图5c中示例(2)不包含左上方相邻区域。具体可对照图5b介绍的内容，在此不做赘述。
可以理解的是，如前述图5b、图5c等示出的图像块中各分区所对应的当前模板仅是示例性的内容，由于几何划分模式包括的分割线类型的不同，该分割线的类型指示了分割线对图像块划分的分割方式，各个分区的当前模板又依据分割线的类型而确定，并且初始当前模板所包含区域对当前模板的最终确定也有影响，因此，各个分区分别对应的当前模板有非常多的种类，在此不一一示出，但均属于本申请所保护的范围。
可见，上述方式中都不直接将初始当前模板作为分区的模板，而对各个分区分别设置分区对应的当前模板，这样对于某个分区而言，可以减小初始当前模板中和另一分区相似的部分对模板匹配计算存在的影响。因此，针对各个分区单独设置每个分区对应的当前模板，由于当前模板是与对应分区相似的部分，在进行模板匹配计算时并没有其他分区的模板信息的参与，基于当前模板能够更好地匹配出相似模板，提高相似模板与当前模板之间的模板匹配度，进而提升预测结果计算的准确度。
可选地，当前模板的类型包括斜边模板和/或非斜边模板。图像块中各个分区对应的当前模板可以包括斜边模板和非斜边模板中的任一种或两种。例如第一分区的当前模板和第二分区的当前模板均属于斜边模板，或者均属于非斜边模板。当所述当前模板的类型为所述斜边模板时，所述斜边模板的斜边位置和斜率根据划分所述图像块的分割线设置。由于图像块使用的分割线是基于拟合的物体边界，通过分割线设置斜边模板的斜边位置和斜率，可以使得基于分割线划分出的当前模板对于图像边缘的贴合度高。斜边模板的斜边边界和分割线所在的直线是重合的，示例性地，如上述图5b中的(1)所示的当前模板均为斜边模板，且当前模板A或者当前模板B的斜边是与分割线所在的直线重合。当所述当前模板的类型为所述非斜边模板时，所述非斜边模板的位置根据所述分割线所在直线经过所述初始当前模板的区域进行设置。分割线所在直线与初始当前模板的上下边界或者左右边界之间存在交点，以交点为起始点，作一条垂线与模板相交于另一点，可以划定该分割线所在直线经过的模板区域，该模板区域既不是第一分区的当前模板，也不是第二分区的当前模板，属于非分区模板区域。示例性地，如前述图5c所示的当前模板，当前模板A和当前模板B均属于非斜边模板，该非斜边模板具体为矩形形状的模板。由于非斜边模板是规整的模板，垂直线比斜线的坐标计算量小，因此可以有效简化分区对应模板的坐标的计算量。
由于至少一个相似模板是与当前模板对应的，因此，相似模板的类型和当前模板的类型也可以相同，例如当前模板属于斜边模板，则相似模板也属于斜边模板。只是相似模板和当前模板在图像中的位置有所不同。
可选地，当前模板还具备以下特点：当前模板的横坐标取值范围与图像块的长和/宽不相等；当前模板纵坐标取值范围与图像块的长和/宽不相等。仍旧以图5b所示出的内容为例，例如图5b中的(1)所示的当前模板B的横坐标取值范围小于图像块的长，当前模板A的纵坐标取值范围大于图像块的宽。这是由于分割线划分而引起的，当前模板的形状根据分割线的位置而设置，可以避免将不必要的采样作为模板，从而使得根据此当前模板查找到的内容和分区的相似度更高，进而提高预测结果的准确度。
相似模板是指与当前模板相似的模板，该相似模板的位置不同于当前模板，但和当前模板具有相同的形状、大小(具体是指图像区域大小)、类型等。因此，相似模板也可以具备以下特点：相似模板的横坐标取值范围与图像块的长和/宽不相等；相似模板纵坐标取值范围与图像块的长和/宽不相等。任一相似模板的类型包括斜边模板和/或非斜边模板。
按照预设策略可以确定图像块各个分区的当前模板各自对应的至少一个相似模板。也即，对于每个分区的当前模板来说，都可以确定出与其对应的至少一个相似模板。由于预设策略的存在，对于图像块中任一分区可匹配更合适该分区的规则来确定相似模板，从而保证相似模板的准确性。
在一个实施例中，S402的实现方式可以包括以下内容：根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板。
基于模板匹配的几何划分模式的类型包括基于模板匹配的帧内预测和/或基于模板匹配的帧间预测。预定搜索范围用于确定相似模板的图像区域。预定搜索范围可以是与基于模板匹配的几何划分模式的类型匹配的搜索范围，例如图像块使用基于模板匹配的帧内预测，图像块为当前编码的CU，预定搜索范围可以是与CU所在的CTU相邻的已编码CTU。可以理解的是，在目标预测模式为基于模板匹配的几何划分模式时，对于图像块中任一个分区对应的当前模板，均可以按照如S402所述的方式确定该分区的当前模板对应的至少一个相似模板。不同分区采用的基于模板匹配的几何划分模式的类型不同(或相同)时，预定搜索范围可以不同(或相同)。
对于不同类型的基于模板匹配的几何划分模式，可以匹配相应的预定搜索范围，并从中确定出当前模板对应的至少一个相似模板，从而使得相似模板与当前模板的适配度更高。可选地，可以基于模板代价计算与当前模板对应的至少一个相似模板，至少一个相似模板是各个相似模板中满足匹配要求的相似模板，匹配要求例如是模板代价满足代价阈值，或者是模板代价最小，当前模板可对应一个相似模板，当前模板也可对应多个(即至少两个)相似模板。
可选地，所述预定搜索范围基于参考块对应的相似模板和/或参考块的预测候选块的位置确定。所述参考块是在图像块所在图像中的已编码块或所述图像块所在的图像对应的参考图像中的图像块。在一实施例中，所述参考块是步骤S301～步骤S302中的图像块或者步骤S401～步骤S403中的图像块的相邻块，或者所述参考块是步骤S301～步骤S302中的图像块或者步骤S401～步骤S403中的图像块对应的初始运动矢量确定的参考图像中的图像块。
换而言之，可以根据参考块对应的相似模板和/或参考块的预测候选块的位置确定预定搜索范围。展开来说，可以根据参考块对应的相似模板的位置确定预定搜索范围，也可以根据参考块的预测候选块的位置确定预定搜索范围，还可以根据参考块对应的相似模板的位置和参考块的预测候选块的位置确定预定搜索范围。此处预定搜索范围是与当前编码的图像块或者图像块中一分区对应的搜索范围，在该搜索范围内可以查找与该图像块对应的相似模板，从而确定出该图像的预测候选块以用于预测得到预测结果。
由于参考块对应的相似模板、预测候选块等均是可以确定的数据，并且参考块和当前编码的图像块是存在关联关系的，因此，基于这些数据可以保证预定搜索范围确定的准确性，并且可以提高确定的效率。
在一种实现方式中，根据参考块对应的相似模板的位置确定预定搜索范围，包括：根据参考块和图像块之间的位置关系、参考块对应的相似模板的位置，确定预定搜索范围。参考块与图像块的位置关系包括相邻关系(例如左相邻、上相邻)，参考块的相似模板的位置可以指示当前块最终的预定搜索范围。举例来说，若参考块为当前块的左相邻块，则在参考块的相似模板的右侧，可以确定一个预定大小的搜索范围来作为当前块的预定搜索范围。若参考块为当前块上相邻的块，则在参考块的相似模板的下方，确定一个预定大小的搜索范围来作为当前块的预定搜索范围。由于参考块与图像块之间的相关性，相似模板也可能是相关的，基于参考块的相似模板的位置锁定当前编码的图像块的预定搜索范围，是一种具备参考性且方便的方式。
在另一种实现方式中，根据参考块的预测候选块的位置确定预定搜索范围，包括：根据参考块和图像块之间的位置关系、参考块的预测候选块的位置，确定预定搜索范围。参考块的预测候选块是指参考块对应的用于预测的候选块。参考块的预测候选块的位置可以是在参考块所在图像中的位置，也可以是在该图像对应的参考图像中的位置，这取决于参考块中各个分区所使用的基于模板匹配的几何划分模式的类型。例如，若基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则预测候选块的位置可以在图像块所在图像的重建图像区域内，若基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则预测候选块的位置可以在图像块所在图像对应的参考图像内。在一实施例中，参考块与图像块的位置关系包括相邻关系。结合不同图像块之间的位置关系和预测候选块的位置可确定出预定搜索范围，举例来说，若参考块为当前块左相邻的块，则在参考块的预测候选块的左侧，确定一个预定大小的搜索范围来作为当前块的预定搜索范围。若参考块为当前块上相邻的块，则在参考块的预测候选块的下方，确定一个预定大小的搜索范围来作为当前块的预定搜索范围。由于预测候选块是与参考块对应的，基于预测候选块同样可以锁定当前块的预定搜索范围，也是一种具备参考性的便捷方式。
在又一种实现方式中，根据参考块对应的相似模板的位置和参考块的预测候选块的位置确定预定搜索范围，包括：根据参考块和图像块之间的位置关系、参考块的预测候选块的位置以及参考块对应的相似模板的位置，确定预定搜索范围。其中所包含的相关释义可以参见前述描述，举例来说，若参考块为当前块左相邻的块，则在预测候选块左侧以及相似模板的右侧，确定一个预定大小的搜索范围来作为当前块的预定搜索范围。这样可以综合关联数据丰富的位置信息等更精准地确定出预定搜索范围。
以上方式中，参考块可以是在图像块所在图像中的已编码块，例如当前编码的图像块相邻的图像块。当一副图像中具有的重复元素较多时，依据参考块最终确定的相似模板和/或预测候选块的位置确定预定搜索范围，可以大大地缩小相似模板的搜索范围，提高搜索效率。参考块也可以是图像块所在的图像对应的参考图像中的图像块，例如图像块所在的图像对应的参考图像可以是在图像块所在图像的前一帧已编码图像，参考帧的数量可以包括一个或多个，参考图像中图像块对应的相似模板的位置均可以作为确定预定搜索范围的参考数据，在此方式下，可以不用统计当前块的运动矢量，在图像块的运动矢量不可用或者丢失时，此方式也可以作为一种备用方式，从而基于已知数据快速地确定出预定搜索范围，提高处理速度。
可选地，预定搜索范围(或者说预定搜索区域)的尺寸可以与图像块的尺寸成正比，以使得每个像素具有固定数量的模板代价的比较结果，例如SAD(Sum ofabsolute differences，一种匹配代价函数，差的绝对值之和)比较结果。预定搜索范围的宽度＝a*BlkW，搜索范围的高度SearchRange_h＝a*BlkH，其中，a是用于控制增益和复杂度之间的平衡的常数。在实际中，a可以取5，BlkW和BlkH分别表示图像块的宽度和高度。本申请中，“*”表示执行乘法计算。
在一种可行的实现方式中，所述预定搜索范围与所述基于模板匹配的几何划分模式的类型匹配：若基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则预定搜索范围为图像块所在的图像；若基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则预定搜索范围为图像块所在的图像对应的参考图像。
基于参考块对应的相似模板和/或参考块的预测候选块的位置可以进一步缩小与基于模板匹配的几何划分模式的类型匹配的预定搜索范围，即在图像块所在的图像或者是图像块所在图像对应的参考图像中锁定更小的搜索范围，并从中确定出相似模板。举例来说，图像块中一分区使用基于模板匹配的帧间预测的几何划分模式，可以确定出预定搜索范围为图像块所在的图像的已编码区域，若参考块最终确定的相似模板/预测候选块的位置在该已编码区域内，则根据参考块和当前块的位置关系以及参考块的相似模板的位置，从该已编码区域内再次确定当前块的预定搜索范围，此时预定搜索范围是缩小后的搜索范围。
S403，根据所述相似模板确定所述图像块的预测结果。
根据图像块中各个分区对应的当前模板的相似模板，确定第一预测结果集合和第二预测结果集合，该第一预测结果集合和第二预测结果集合可以是基于对应分区的相似模板定位的预测候选块，之后可以根据分区的权重值集合，对第一预测结果集合和第二预测结果集合进行融合，得到图像块的预测结果。第一预测结果集合和第二预测结果集合中的至少一个可以是通过基于模板匹配的帧内预测模式而确定的。
在一个实施例中，S403的实现方式包括如下内容：根据对应分区的至少一个相似模板确定该分区的至少一个预测候选块；根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果。
由于图像块的每个分区都对应有当前模板，对于每个分区的当前模板都可以按照上述方式确定出至少一个相似模板，因此，相似模板可以是对于分区而言的，每个分区的当前模板对应的相似模板的数量可以相同或者不同，例如第一分区的当前模板对应3个相似模板，第二分区的当前模板对应2个相似模板。对应分区可以是图像块的任一分区，对于图像块中任一分区，可以根据任一分区的至少一个相似模板确定出该分区的至少一个预测候选块，一个相似模板对应一个预测候选块。举例来说，第一分区的3个相似模板可以确定出3个预测候选块。基于预测候选块与相似模板之间的相邻关系，可以将与相似模板相邻的左侧或者下侧区域中，与图像块形状、大小相等的图像区域作为预测候选块，该预测候选块是一个重建图像块。
在一种实现方式中，根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果，包括：根据预设权重对对应分区的至少一个预测候选块进行加权处理，得到所述图像块对应分区的目标预测候选块；根据分区权重和所述图像块中各个分区的目标预测候选块，确定所述图像块的预测结果。
对于图像块的至少一个分区中的每个分区，可以确定至少一个预测候选块，基于预设权重对该分区的至少一个预测候选块进行加权处理，进而得到关于该分区最终的预测候选块(即目标预测候选块)。每个预测候选块都配置有一个预设权重，各个预测候选块中的像素值可以和预设权重相乘求和得到目标预测候选块。举例来说，对于图像块中的分区A，至少一个预测候选块包括预测候选块A1和预测候选块A2，利用预测候选块A1以及预测候选块A2可以得到关于分区A的最终预测候选块，即：
目标预测候选块＝预测候选块A1*W1+预测候选块A2*W2。
其中，W1和W2为预设权重。同理，也可以通过上述方式，对分区B对应的至少一个预测候选块(包括预测候选块B1和预测候选块B2)，得到关于分区B的最终预测候选块。通过确定至少一个预测候选块，并采用加权的形式来得到最终的预测候选块，综合参考图像中的已知信息，可以使得预测的精度更高，对应的残差越小，从而有利于提高压缩效率。
可选地，每个预测候选块配置有预设权重，所述预设权重的设置策略包括以下至少一种：根据所述预测候选块与所述图像块之间的距离设置；根据所述预测候选块的数量设置；根据权重更新标志设置。
在一个实现方式中，预设权重的设置策略包括：根据传输于已编码图像块中的权重更新标志设置。权重更新标志wight_flag可以传输于CTU或者是CU中，该权重更新标志可以指示对于该CTU或CU采用更新的权重。举例来说，若权重更新标志为1，则指示该CTU或CU采用更新的权重，和/或，若权重更新标志为0，则权重不更新。
如果权重更新标志不存在于比特流中，则默认权重更新标志为0，该CTU或CU不采用更新的权重。若权重更新标志为1，则在解码端进一步解析权重值。新解析出的权重值可用于当前CTU或CU，也可用于当前CTU或CU编码顺序后的CTU或CU，以及可以用于利用四叉树划分当前CTU或CU得到的CU。
在另一种实现方式中，预设权重的设置策略包括：根据预测候选块距离图像块之间的距离设置。例如与图像块中一分区距离近的预测候选块的预设权重可以大于与该分区距离远的预测候选块的预设权重，此时，不同预测候选块对应的预设权重可能不相等或者是相等。举例来说，若预设权重W1为0.6，W2为0.4，则预设权重W1为与分区空间位置距离较近的预测候选块的权重，预设权重W2为与分区空间位置距离较远的预测候选块的权重。在此规则下，通过预设权重的解析，也可以判断出对应预测候选块与分区距离的远近。通常情况下，距离分区较近的预测候选块与分区的相似度较高，通过对空间位置距离较近的预测候选块分配较大的权重，可以提供预测的准确度。
在又一种实现方式中，预设权重的设置策略包括：根据所述预测候选块的数量设置。此处预测候选块的数量是指当前模板对应的至少一个预测候选块的数量。可以基于预测候选块的数量对各个预测候选块设置相同的权重。举例来说，在上述示例中，若通过两个预测候选块来确定目标预测候选块，则可以设置两个预测候选块分别对应的预设权重W1＝W2＝0.5,，若通过三个预测候选块来确定目标预测候选块，则可以设置预设权重W1＝W2＝W3＝0.33。也可以基于预测候选块的数量对各个预测候选块设置不同的权重，例如预测候选块对应的相似模板与当前模板相似度高的，该预测候选块设置的权重更大。
预设权重可以是预先设置的，也可以是基于权重更新标志来更新的。基于权重更新标识可以解析预设权重，举例来说，当wight_flag为1时，解析出预设权重W1和W2，可以将该预设权重应用于当前CTU或者CU。当权重更新标志wight_flag为1时，在使用相等数值权重的情况下，首先可以解析出预设权重数目wight_number，若预设权重数目wight_number为2，则存在2个预设权重，且各个预设权重均为0.5，若预设权重数目wight_number为4，则存在4个预设权重，且各个预设权重为0.25，进一步可以将这些预设权重作为当前块的预测候选块的权重来使用。
在一个可行的实施例中，如上介绍的基于预测候选块确定预测结果的方式，也可以直接作为如图3a对应实施例中对应S301的具体实现方式。即，根据目标预测模式确定图像块的预测结果的实现方式可以是：根据所述目标预测模式和预设策略，确定所述图像块的至少一个分区中的每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
详细地说，可以根据所述图像块的至少一个分区中的每一个分区对应的至少一个预测候选块的各自的权重对所述至少一个预测候选块进行加权处理，得到每一个分区对应的加权处理后的预测候选块，以用于确定所述图像块的预测结果；其中，所述目标预测模式用于确定所述至少一个分区，所述预设策略用于确定所述至少一个分区各自对应的至少一个相似模板。此处的权重同上述介绍的预设权重，对应的处理方式可参考前述介绍的内容，在此不做赘述。
本申请实施例提供的图像处理方法，可以在目标预测模式包括基于模板匹配的几何划分模式的情况下，通过预设策略确定图像块中各个分区对应当前模板的至少一个相似模板，从而基于相似模板确定图像块的预测结果。由于预设策略可指示与图像块的当前模板匹配的相似模板的确定准则，该预设策略可以与对应分区所使用的基于模板匹配的几何划分模式相适配，提升相似模板与当前模板之间的匹配度，有助于提高预测准确度。在此基础上，图像块各个分区也可以使用相同或不同类型的基于模板匹配的几何划分模式，预设策略具体与该基于模板匹配的GPM模式的类型适配，进而可确定采用对应类型的基于模板匹配的GPM模式的分区的相似模板，该相似模板是匹配度更佳的相似模板，基于匹配度更高的相似模板可以得到更准确的预测结果。
第三实施例
请参阅图6，图6是根据第三实施例示出的一种图像处理方法的流程示意图，该实施例中的执行主体可以是一个计算机设备或者是多个计算机设备构成的集群，该计算机设备可以是智能终端(如前述移动终端100)，也可以是服务器，此处，以本实施例中的执行主体为智能终端为例进行说明。
在图4a所介绍的实施例的基础之上，根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板的可选实现方式包括以下S601和/或S602。需要说明的是，对于图像块的至少一个分区(包括第一分区和第二分区)中的每个分区，都可以采用如下述S601或S602的方式来确定该分区的当前模板对应的至少一个相似模板。
S601，若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则从所述图像块所在的图像中确定当前模板对应的至少一个相似模板。
针对图像块中任一分区，若该分区使用的基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，说明使用基于模板匹配的几何划分模式的分区可以参考帧内预测的处理方式匹配出相应的相似模板，具体可以从图像块所在的图像中确定该分区的当前模板对应的至少一个相似模板。基于模板匹配的帧内预测是一种特殊的帧内预测模式，可简称为模板帧内预测。模板帧内预测可以在当前图像中确定分区的当前模板对应的相似模板。
基于模板匹配的帧内预测是从当前图像(即当前块所在的图像)的重建部分复制预测候选块，预测候选块的相似模板与当前模板相匹配。在当前图像的重建部分的预定的范围内，可以搜索当前模板的最佳相似模板，并将最佳相似模板作为使用基于模板匹配的帧内预测的分区的当前模板对应的相似模板。在编码器中，编码器可以发送一标志指示基于模板匹配的帧内预测的使用情况。此处的使用情况是指基于模板匹配的帧内预测被图像块哪个分区使用，在解码器中，解码器会解析该标志而决定是否启用基于模板匹配的帧内预测。举例来说，在编码端图像块第一分区使用了基于模板匹配的帧内预测，而第二分区未使用，在解码端通过解析出来的标志决定启用基于模板匹配的帧内预测，并将基于模板匹配的帧内预测应用于图像块的第一分区。
在一个实施例中，S601的实现内容包括：按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板；根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
图像块所在的图像中的重建图像区域是已编码区域，该重建图像区域可以作为预定搜索范围，并从中搜索出至少一个相似模板，至少一个相似模板是当前模板的最佳相似模板。示例性地，请参见图7a所示出的图像块所在图像中的重建图像区域，包括R1、R2、R3、R4。R1是图像块CU左上角位置相邻的当前CTU内的重建区域，R2是当前CTU左上角位置相邻的重建区域，R3是与CTU上边界相邻的重建区域，R4是与当前CTU左边界相邻的重建区域。在这些重建区域中可以按照预设搜索规则搜索与当前模板大小相同的区域，确定出与当前模板匹配的相似模板，进而从所匹配的相似模板中确定出当前模板的至少一个相似模板。如图7a所示，在重建区域R1中确定出与第一分区的当前模板A对应的相似模板A1，以及在重建区域R2中确定出与第一分区的当前模板A所匹配的相似模板A2，在重建区域R3中确定出与第二分区的当前模板B所匹配的相似模板B1，以及在重建区域R2中确定出与第二分区的当前模板B所匹配的相似模板B2。这些与对应当前模板所匹配的相似模板都与对应的当前模板有相同的形状。
预设搜索规则是指在重建图像区域中搜索出与当前模板匹配的相似模板的规则，该预设搜索规则例如是在搜索到指定数量的相似模板时停止搜索，或者是指定搜索方式。通过预设搜索规则的设定，可以从图像块所在的图像中的重建图像区域中 有规律的查找与当前模板匹配的相似模板，可基于预设搜索规则的具体内容设定，在一定程度上提升查找效率和/或查找全面性。
在一种实现方式中，预设搜索规则包括以下至少一种：①在对在所述图像块所在的图像中的重建图像区域进行搜索的过程中，当出现预设数量个满足匹配条件的目标区域，则将各个满足匹配条件的目标区域确定为与当前模板匹配的相似模板；②当所述重建图像区域搜索完毕之后，将满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板。
对于预设搜索规则①，可以在重建图像区域搜索的过程中，当出现第一个满足匹配条件的目标区域，可以将该区域作为与当前模板匹配的相似模板，当出现第二个满足匹配条件的目标区域，也可以将该区域作为与当前模板匹配的相似模板，这些目标区域是与当前编码的图像块形状大小相同的重建块。按照此种方式，在出现预设数量个满足匹配条件的目标区域，无论重建图像区域是否搜索完毕，均可以停止搜索，并得到各个满足匹配条件的目标区域作为当前模板匹配的各个相似模板。这样，对于相似模板的搜索可能无需搜索完重建图像区域就可确定出与当前模板匹配的相似模板，搜索效率高，搜索所需的资源少，由于所匹配的相似模板是满足匹配条件的，因此可以在保证后续预测结果精确度的同时提高整体编码效率。对于预设搜索规则②，在重建图像区域搜索的过程中，可以将各个区域与当前块的匹配度记录下来，等到重建图像区域搜索完毕之后，再将记录的各个区域的匹配度中满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板。举例来说，当搜索完重建图像区域之后，出现满足匹配条件的两个(预设数量即为2)目标区域，那么可以将这两个目标区域均作为当前模板匹配的相似模板。这样可以对整个重建图像区域进行完整全面的搜索，并通过全面的比对确定出与当前模板匹配的相似模板，有利于提供匹配度高的相似模板，从而有助于提升预测结果的精确性。
预设数量个是对与当前模板匹配的相似模板的数量限定，可以是1个，或者是2个，或者是3个，等等，在此不做限制。满足匹配条件可以是指目标区域与图像块之间的匹配度符合要求，该匹配度可以通过目标区域与图像块之间的模板代价来衡量，匹配度符合要求可以是模板代价最小，或者是模板代价小于等于模板代价阈值。可选地，模板代价可以是当前模板与所搜索的模板之间的SSD(Sum of Squared Difference，差值均方和)，也可以是SAD(Sum of Absolute Difference，绝对误差和)，或者是其他计算函数。模板代价小，说明匹配度高，所搜索的模板与当前模板之间的相似度大，模板代价大，说明匹配度低，所搜索的模板与当前模板之间的相似度小。
举例来说，可以在一个或多个预定范围内搜索一个相似模板，使得该相似模板与当前模板之间的SAD最小。在搜索过程中，对于与当前模板大小相同的区域，可以计算该区域与当前模板之间的SSD，可以将搜索出现SSD小于阈值的目标区域作为与当前模板匹配的相似模板，或者是将搜索出现的SSD最小的目标区域作为与当前模板匹配的相似模板。通过SSD值的大小可以确定该区域和当前模板之间的相似程度，如果在该SSD小于一阈值，可以认为该区域为与当前模板匹配的相似模板，当前模板与相似模板之间的相似程度较高。
在一种实现方式中，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板，包括以下内容：确定所匹配的相似模板与所述当前模板之间的模板代价，将满足预设模板条件的模板代价对应的所匹配的相似模板确定为所述当前模板对应的至少一个相似模板。
在与当前模板匹配的相似模板的数量包括多个的情况下，针对多个所匹配的相似模板中的每一个相似模板，均可以计算所匹配的相似模板与当前模板之间的模板代价。模板代价可以表示率失真代价或者是模板之间的相似度，模板之间的相似度可以采用所匹配的相似模板与当前模板之间的差值均方和SSD来衡量，也可以采用例如绝对误差和SAD，或者是其他方式来衡量。可选地，所述满足预设模板条件包括以下至少一种：模板代价最小，模板代价小于或等于预设代价阈值。该预设代价阈值是用于判别所匹配的相似模板是否为当前模板的最佳相似模板。模板代价最小，说明所匹配的相似模板和当前模板是匹配度最高的，将其作为最佳相似模板可以提高预测结果的准确性。模板代价小于或等于预设代价阈值，说明所匹配的相似模板和当前模板是匹配度较高，也可以将其作为当前模板的最佳相似模板，此时最佳相似模板的数量可能包括多个，确定预测结果的过程中参考的信息更多。可以将最小模板代价对应的与当前模板匹配的相似模板确定为当前模板最终对应的相似模板，也可以将小于或等于预设代价阈值的模板代价对应的与当前模板匹配的相似模板确定为当前模板最终对应的相似模板。需要说明的是，前述预设搜索规则中也可模板代价阈值选定与当前模板匹配的相似模板，为保证从所匹配的相似模板中筛选出匹配度更高的相似模板，此处设置的预设代价阈值大于前述的模板代价阈值。
通过这样的方式，可以从初始筛选出来所匹配的相似模板中进一步进行筛选，可以基于模板代价对相似度模板是否满足预设模板条件进行再次判别，以从中确定出匹配度更高的相似模板，提高相似模板与当前模板之间的相似度，减小预测过程中的残差，进而得到更还原输入的原始图像块的预测结果。
在一个实施例中，相似模板可用于确定图像块的预测结果。可以按照如图4a实施例中S403所介绍的实现方式。在图像块中一分区使用基于模板匹配的帧内预测时，可以基于相似模板确定预测候选块，可以从中确定出一个预测候选块直接作为图像块的预测结果，或者是从中确定至少一个预测候选块进行加权求和之后，确定图像块的预测结果。
可以理解的是，此处当前模板为图像块中使用基于模板匹配的帧内预测的分区所对应的当前模板。当前模板相关介绍可以参见前述图4a实施例中所介绍的内容。示例性地，此处以当前模板为非斜边模板为例，且第一分区和第二分区均使用基于模板匹配的帧内预测，如图7b示出了基于模板匹配的帧内预测的示意图。如图7b所示，在当前块所在图像的重建图像区域中确定出当前模板A对应的相似模板A1和相似模板A2，当前模板B对应的相似模板B1和相似模板B1，基于这些相似模板可分别确定出对应的预测候选块，包括预测候选块A11、预测候选块A22、预测候选块B11以及预测候选块B22，之后再基于这几个预测候选块可以进一步确定出图像块的预测结果。
S602，若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板。
针对图像块中任一分区，若该分区使用的基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，说明使用基于模板匹配的几何划分模式的分区可以参考帧间预测的处理方式匹配出相应的相似模板，具体可以从图像块所在的图像对应的参考图像中确定该分区的当前模板对应的至少一个相似模板。
在一个实施例中，S602包括以下方式一和方式二：
方式一：根据至少一个初始运动矢量从所述图像块所在的图像对应的参考图像中确定至少一个初始相似模板，在包括任一个所述初始相似模板的预定范围内搜索与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
对于图像块中的任一分区，在该分区使用基于模板匹配的帧间预测(可简称为模板帧间预测)时，可以基于运动矢量在参考图像中确定当前模板对应的相似模板。首先，可以从运动矢量候选列表中选取一个或多个运动矢量作为初始运动矢量，然后，根据选取的初始运动矢量可以确定初始相似模板，接着，可以在包括初始相似模板的预定范围内搜索与当前模板相匹配的相似模板，当运动矢量只有一个时，所匹配的相似模板即为最佳相似模板，可以直接将其确定为当前块对应的一个相似模板，并根据该相似模板的位置确定对应的预测候选块。示例性地，请参见图7c所示的基于模板匹配的帧间预测，分区A采用基于模板匹配的帧间预测，基于初始运动矢量MV1在参考图像1中搜索与当前模板A最匹配的相似模板A1，并基于该相似模板A1确定出预测候选块A11。
优选地，也可以在运动矢量列表中选取多个运动矢量分别作为初始运动矢量mv1，mv2，...，mvn(n大于等于2)。然后，根据初始运动矢量mv1，mv2，...，mvn分别确定n个初始相似模板。在根据n个初始相似模板在预定范围内搜索与当前模板A相匹配的n个相似模板，这些相似模板为较佳相似模板，最后，在所匹配的n相似模板中确定最佳相似模板，并将最佳相似模板作为当前模板对应的相似模板。通过多个所匹配的相似模板的筛选，可以在更大的选择范围内为当前模板更全面地挑选最匹配的相似模板。
在一个可行的实施例中，基于模板匹配的帧间预测使用的运动矢量候选列表的构建过程如下。如图7d所示为运动矢量候选列表的相邻块位置示意图，图7e为运动矢量候选列表。空域合并候选列表最多可选出4个候选运动矢量，构造顺序为：上侧相邻块B1、左侧相邻块A1、右上侧相邻块B0、左下侧相邻块A0、左上侧相邻块B2，参考图像对应位置块Col的运动矢量依次排列，并将这些运动矢量作为当前块的运动矢量候选。需要说明的是，其他位置不可用时才考虑B2。在添加左下侧相邻块A0之后，需要进行冗余检测，以确保列表中不具有相同运动矢量候选。此外，合并候选列表中还可以添加历史参考块his，第一候选运动矢量、第二候选运动矢量的平均运动矢量avg，以及零运动矢量0。
方式二：所述参考图像包括第一参考图像和/或第二参考图像，从所述图像块所在的图像对应的第一参考图像中确定当前模板对应的第一相似模板，和/或，从所述图像块所在的图像对应的第二参考图像中确定当前模板对应的第二相似模板，将所述第一相似模板和/或所述第二相似模板确定为所述当前模板对应的至少一个相似模板。
图像块中任一分区采用基于模板匹配的帧间预测，可以在参考图像中确定相似模板。参考图像可以包括第一参考图像和第二参考图像，此时，可以在当前块所在的图像的前后两帧参考图像中确定相似模板，也可以在一帧参考图像中确定相似模板。即可以从第一参考图像中确定当前模板对应的第一相似模板以及从第二参考图像中确定当前模板对应的第二相似模板，并且第一相似模板和第二相似模板为当前模板对应的至少一个相似模板。也可以仅从第一参考图像中确定第一相似模板，并将该第一相似模板作为当前模板对应的至少一个相似模板。还可以仅从第二参考图像中确定第二相似模板，并将该第二相似模板作为当前模板对应的至少一个相似模板。图像块所在的图像的帧类型可以为I帧或者是P帧，并基于不同的帧类型从第一参考图像和/或第二参考图像中确定相似模板。举例来说，图像块为P Slice(单向帧间编码的条带)，则可以从一个参考图像中确定，例如从第一参考图像或者第二参考图像中确定当前模板对应的至少一个相似模板。图像块为B Slice(双向帧间编码的条带)，则可以从两个参考图像，包括第一参考图像和第二参考图像中确定当前模板对应的至少一个相似模板。对不同帧类型对应不同的帧间编码方式匹配相适应的参考图像，可以更有效地利用相关资源得到当前模板对应的至少一个相似模板，实现对图像块的高效压缩或解压缩。需要说明的是，从所述图像块所在的图像对应的第一参考图像(或者第二参考图像)中确定当前模板对应的第一相似模板(第二相似模板)，均可以采用如方式一所描述的内容，即通过运动矢量确定相似模板。
示例性地，请参见图7f所示出的另一种基于模板匹配的帧间预测的示意图。分区B采用基于模板匹配的帧间预测，可以基于初始运动矢量MV1在参考图像1中确定出当前模板B对应的相似模板B1，并基于该相似模板B1确定出预测候选块B11，基于初始运动矢量MV2在参考图像2中确定出当前模板B对应的相似模板B2，并基于该相似模板B2确定出预测候选块B22。
在一种实施例中，若图像块中第一分区和第二分区采用的基于模板匹配的几何划分模式的类型相同，例如第一分区和第二分区均采用基于模板匹配的帧内预测，则各个分区可按照S601的方式确定出对应分区的至少一个相似模板；例如第一分区和第二分区均采用基于模板匹配的帧间预测，则可按照S602的方式确定出对应分区的至少一个相似模板。如图7g所示的各个分区采用基于模板匹配的帧间预测的示意图，如图7g所示，当前图像中当前块的两个分区均是采用基于模板匹配的帧间预测，基于初始运动矢量MV1可以在参考图像1中确定出当前模板A对应的相似模板A1，并基于该相似模板A1确定出预测候选块A11，基于初始运动矢量MV2在参考图像2中确定出当前模板B对应的相似模板B1，并基于该相似模板B1 确定出预测候选块B11，之后，再基于预测候选块A11和预测候选块B11确定出当前块的预测结果。
需要说明的是，以上S601～S602所描述的内容也可以直接作为通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板的具体实现方式。
可以理解的是，若图像块中第一分区和第二分区采用的基于模板匹配的几何划分模式的类型不同，例如第一分区采用基于模板匹配的帧内预测，第二分区采用基于模板匹配的帧间预测，则第一分区可按照S601的方式确定出第一分区的至少一个相似模板，第二分区可按照S601的方式确定出第二分区的至少一个相似模板。反之，第一分区采用基于模板匹配的帧间预测，第二分区采用基于模板匹配的帧内预测，则第一分区可按照S602的方式确定出第一分区的至少一个相似模板，第二分区可按照S601的方式确定出第二分区的至少一个相似模板。
示例性地，请参见图7h所示出的采用的基于模板匹配的几何划分模式的类型不同时的预测示意图。如图7h，当前图像中当前块的分区B采用基于模板匹配的帧间预测，分区A使用基于模板匹配的帧内预测。具体地，当前块的分区B采用的基于模板匹配的帧间预测为双向预测模式。在此模式中，分区B存在两个运动矢量，即初始运动矢量MV1和初始运动矢量MV2。基于初始运动矢量MV1在参考图像1中确定出当前模板B对应的相似模板B1，基于初始运动矢量MV2在参考图像2中确定出当前模板B对应的相似模板B2，相似模板B2和相似模板B1均可作为当前模板的至少一个相似模板，进而基于相似模板B1确定出预测候选块B11，基于相似模板B2确定出预测候选块B22，预测候选块B11和B22可以进一步加权求和得到分区B对应的目标预测块。分区A采用基于模板匹配的帧内预测，在当前CTU相邻的重建区域内，搜索出与当前模板A对应的相似模板A1和相似模板A2，进而基于相似模板A1确定出预测候选块A11，以及基于相似模板A2确定出预测候选块A21，预测候选块A11和预测候选块A21可以进一步加权求和得到分区A对应的目标预测块，最后可以基于分区权重对两个分区的目标预测块进行加权，得到图像块的预测结果。需要说明的是，在本发明的其他实施方式中，基于模板匹配的帧间预测可以为单向预测模式。也就是说，若一分区采用单向预测模式，则该分区仅存在一个运动矢量。此外，在其他实施方式中，当前图像中当前块的分区A可以采用基于模板匹配的帧间预测，分区B可以采用基于模板匹配的帧内预测。
由此，在基于模板匹配的GPM模式中，支持使用基于模板匹配的帧间预测和基于模板匹配的帧内预测这两种类型。当图像块使用基于模板匹配的GPM模式，图像块各个分区可以灵活地组合不同或相同类型的基于模板匹配的GPM模式，丰富了对同一图像块的预测处理方式，使得预测更加灵活。图像块各分区使用的基于模板匹配的GPM模式的类型不同时，不仅可以在参考图像中搜索匹配的相似模板，也可以在当前图像的已编码区域中搜索匹配的相似模板，从而可以更好地搜索出最佳匹配的相似模板，有利于提升图像块的预测结果的精确性，减小图像失真，以更好地还原真实图像。
本申请实施例提供的图像处理方法，对于基于模板匹配的几何划分模式的不同类型，采用了不同的预测处理方式得到当前模板对应的至少一个相似模板。该预测处理方式是与基于模板匹配的几何划分模式的类型相匹配的，从而可以更好地针对每个分区搜索匹配度更佳的相似模板，基于匹配度更佳的相似模板可为预测结果提供更准确的处理依据。
第四实施例
请参阅图8，图8是根据第四实施例示出的一种图像处理方法的流程示意图，该实施例中的执行主体可以是一个计算机设备或者是多个计算机设备构成的集群，该计算机设备可以是智能终端(如前述移动终端100)，也可以是服务器，此处，以本实施例中的执行主体为智能终端为例进行说明。
S801，获取或确定图像块的目标预测模式。
若目标预测模式为基于模板匹配的几何划分模式，则根据目标预测模式确定图像块的预测结果的实现方式可以包括S802所描述的内容。或者说，根据目标预测模式和预设策略确定图像块的预测结果的实现方式可以包括S802所描述的内容。
S802，确定所述图像块的初始当前模板，以用于通过所述初始当前模板确定所述图像块的预测结果。
图像块的初始当前模板包括与图像块相邻的已编码图像区域。可选地，所述初始当前模板包括分区模板区域和非分区模板区域。分区模板区域包括第一分区对应的当前模板以及第二分区对应的当前模板。
在一种实现方式中，所述初始当前模板包括与所述图像块的上边界相邻的区域、与所述图像块的左边界相邻的区域、与所述图像块左上方的相邻区域中的至少一个区域；所述初始当前模板包括所述第一分区对应的当前模板和所述第二分区对应的当前模板。具体可以参见前述图5a所示出的初始当前模板。
各个分区对应的当前模板具备以下特点：所述第一分区对应的当前模板的横坐标范围与所述第二分区对应的当前模板的横坐标范围不同；或者，所述第一分区对应的当前模板的纵坐标范围与所述第二分区对应的当前模板的纵坐标范围不同；或者，所述第一分区对应的当前模板的长和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的长不相等；或者，所述第一分区对应的当前模板的宽和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的宽不相等。
在一个实现方式中，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界，所述至少一边界位于所述图像块使用的分割线所在的直线上；或者，所述初始当前模板包括所述图像块的第一分区对应的当前模板、所述图像块的第二分区对应的当前模板、以及非分区模板区域，所述图像块使用的分割线所在的直线经过所述非分区模板区域；或者所述非分区模板区域与所述图像块的第一分区对应的当前模板与所述图像块的第二分区对应的当前模板中的至少一个相邻。具体可以参见前述图5b所示出各个分区的当前模板。对于以上内容的详细介绍，可以参见图4a对应实施例中的相关内容。
通过初始当前模板可确定图像块的预测结果，在确定过程中，首先可通过对图像块的初始当前模板进行划分可得到各个分区各自对应的当前模板，在一种实现方式中，可以通过所述图像块使用的分割线所在的直线，将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和所述图像块的第二分区对应的当前模板；其中，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界位于所述图像块使用的分割线所在的直线上。然后可以基于每个分区所使用的基于模板匹配的GPM模式的类型，从图像块所在的图像中确定该分区的当前模板对应的至少一个相似模板，或者是从图像块所在的图像对应的参考图像中确定分区的当前模板对应的至少一个相似模板，进而根据至少一个相似模板确定预测候选块，最终确定出图像块的预测结果。相关步骤中更详细的方式可以参见前述实施例中的相关介绍，在此不做赘述。
此方式下，通过确定初始当前模板，可以首先确定出用于预测处理过程中用于匹配的相似模板，以便于利用该初始当前模板更准确地确定预测结果。
在另一实施例中，根据目标预测模式确定图像块的预测结果的实现方式也可以采用以下S803所描述的内容。或者说，根据目标预测模式和预设策略确定图像块的预测结果的实现方式可以包括S803所描述的内容。
S803，根据所述目标预测模式和预设策略，确定所述图像块的至少一个分区中的每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
可选地，目标预测模式包括基于模板匹配的几何划分模式。在此预测模式下，图像块被划分为至少一个分区，包括第一分区和第二分区。可以确定出图像块各个分区加权后的预测候选块，进而基于预测候选块确定图像块的预测结果。其中，预设策略用于指示在基于模板匹配的几何划分模式下进行模板匹配的准则，通过此预设策略可查找出与图像块的当前模板匹配的相似模板。
在一种实现方式中，根据目标预测模式确定图像块的预测结果，可以包括以下实施步骤：根据所述目标预测模式和预设策略，确定所述图像块的至少一个分区中的每一个分区对应的至少一个预测候选块；对所述至少一个预测候选块进行加权处理，得到所述每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
对于图像块中的每个分区，都可以确定出至少一个预测候选块，每个预测候候选块可配置一预设权重。对于图像块中的任一分区，都可以基于该预设权重对各个预测候选块进行加权处理，得到该分区加权后的预测候选块，为便于描述，可称为目标预测候选块。每个分区都对应一个目标预测候选块，图像块的分区包括第一分区和第二分区，基于第一分区的目标预测候选块和第二分区的预测候选块可确定出该图像块的预测结果。
在一种实现方式中，确定预测候选块的过程包括：先确定出图像块的初始当前模板，并基于初始当前模板确定图像块各个分区的当前模板，然后，在确定各个分区的当前模板对应的多个相似模板，最终基于多个相似模板分别确定至少一个预测候选块，一个相似模板可确定一个预测候选块。需要说明的是，此过程更详细的实现内容可以参见前述实施例中的相关介绍，在此不做赘述。
此方式下，基于目标预测模式和预设策略可以灵活、准确地确定出图像块的预测候选块，从而有助于提升图像块的预测结果的准确度。
需要说明的是，以上步骤S802可以和S801组合为一个方案用于确定图像块的预测结果，步骤S803也可以和S801组合为一个方案用于确定图像块的预测结果。这两个方案可以是并列的方案。
请参见图9，图9是本申请实施例示出的一种图像处理装置的结构示意图，该图像处理装置可以是运行于服务器中的一个计算机程序(包括程序代码)，例如图像处理装置为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。该图像处理装置900包括：获取模块901和确定模块902。
获取模块901，用于获取或确定图像块的目标预测模式；
确定模块902，用于根据所述目标预测模式确定所述图像块的预测结果。
可选地，所述目标预测模式包括基于模板匹配的几何划分模式，所述图像块在所述基于模板匹配的几何划分模式下被划分为第一分区和第二分区。
可选地，包括以下至少一项：所述基于模板匹配的几何划分模式的类型包括：基于模板匹配的帧内预测和/或基于模板匹配的帧间预测；所述第一分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测；所述第二分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测。
在一个实施例中，所述目标预测模式包括基于模板匹配的几何划分模式，确定模块902，用于：通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板；根据所述相似模板确定所述图像块的预测结果。
在一个实施例中，确定模块902，用于：根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板。
可选地，所述预定搜索范围基于参考块对应的相似模板和/或参考块的预测候选块的位置确定。
在一个实施例中，确定模块902，还用于：根据所述参考块和所述图像块之间的位置关系、所述参考块对应的相似模板的位置，确定所述预定搜索范围；和/或，根据所述参考块和所述图像块之间的位置关系、所述参考块的预测候选块的位置，确定所述预定搜索范围。
在一个实施例中，确定模块902，用于：若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则 从所述图像块所在的图像中确定当前模板对应的至少一个相似模板；和/或，若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板。
在一个实施例中，确定模块902，用于：按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板；根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
在一个实施例中，确定模块902，用于：从所述图像块所在的图像对应的第一参考图像中确定当前模板对应的第一相似模板，和/或，从所述图像块所在的图像对应的第二参考图像中确定当前模板对应的第二相似模板，将所述第一相似模板和/或所述第二相似模板确定为所述当前模板对应的至少一个相似模板；和/或，根据至少一个初始运动矢量从所述图像块所在的图像对应的参考图像中确定至少一个初始相似模板，在包括任一个所述初始相似模板的预定范围内搜索与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
可选地，包括以下至少一项：确定模块902，用于：在对在所述图像块所在的图像中的重建图像区域进行搜索的过程中，当出现预设数量个满足匹配条件的目标区域，则将各个满足匹配条件的目标区域确定为与当前模板匹配的相似模板；和/或，当所述重建图像区域搜索完毕之后，将满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板。
可选地，确定模块902，用于：确定所匹配的相似模板与所述当前模板之间的模板代价，将满足预设模板条件的模板代价对应的所匹配的相似模板确定为所述当前模板对应的至少一个相似模板。
在一个实施例中，确定模块902，用于：根据对应分区的至少一个相似模板确定该分区的至少一个预测候选块；根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果。
在一个实施例中，确定模块902，用于：获取所述图像块的初始当前模板；将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板。
在一个实施例中，获取模块901，用于以下至少一项：通过第一标志确定所述图像块的目标预测模式；将最小率失真代价对应的预测模式确定为所述图像块的目标预测模式。
可选地，若所述目标预测模式是基于模板匹配的几何划分模式，则在确定所述预测结果的过程中，利用第一标志指示所述图像块使用所述基于模板匹配的几何划分模式，以及利用至少一第二标志指示所述图像块的第一分区和/或第二分区各自使用的预测处理。
在一个实施例中，确定模块902，用于：根据所述目标预测模式和所述预设策略，确定所述图像块的至少一个分区中的每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
在一个实施例中，若所述目标预测模式为基于模板匹配的几何划分模式，则确定模块902，用于：确定所述图像块的初始当前模板，以用于通过所述初始当前模板确定所述图像块的预测结果；可选地，所述初始当前模板包括分区模板区域和非分区模板区域。
可选地，分区模板区域包括第一分区对应的第一当前模板以及第二分区对应的第二当前模板。
可选地，初始当前模板包括与所述图像块的上边界相邻的区域、与所述图像块的左边界相邻的区域、与所述图像块左上方的相邻区域中的至少一个区域；所述初始当前模板包括所述第一分区对应的当前模板和所述第二分区对应的当前模板。
可选地，所述第一分区对应的当前模板的横坐标范围与所述第二分区对应的当前模板的横坐标范围不同；或者，所述第一分区对应的当前模板的纵坐标范围与所述第二分区对应的当前模板的纵坐标范围不同；或者，所述第一分区对应的当前模板的长和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的长不相等；或者，所述第一分区对应的当前模板的宽和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的宽不相等。
可选地，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界，所述至少一边界位于所述图像块使用的分割线所在的直线上；或者，
所述初始当前模板包括所述图像块的第一分区对应的当前模板、所述图像块的第二分区对应的当前模板、以及非分区模板区域，其中所述图像块使用的分割线所在的直线经过所述非分区模板区域；或者所述非分区模板区域与所述图像块的第一分区对应的当前模板与所述图像块的第二分区对应的当前模板中的至少一个相邻。
可选地，所述当前模板的类型包括斜边模板和/或非斜边模板，所述至少一个相似模板的类型和所述当前模板的类型相同；当所述当前模板的类型为所述斜边模板时，所述斜边模板的斜边位置和斜率根据划分所述图像块的分割线设置；当所述当前模板的类型为所述非斜边模板时，所述非斜边模板的位置根据所述分割线所在直线经过所述初始当前模板的区域进行设置。
本申请实施例还提供一种智能终端，包括存储器、处理器，存储器上存储有图像处理程序，该图像处理程序被处理器执行时实现上述任一实施例中的图像处理方法的步骤。该智能终端可以是如图1所示的移动终端100。
在一可行的实施例中，如图1所示的移动终端100的处理器110可以用于调用存储器109中存储的图像处理程序，以执行如下操作：S1：获取或确定图像块的目标预测模式；S2：根据所述目标预测模式确定所述图像块的预测结果。
可选地，所述目标预测模式包括基于模板匹配的几何划分模式，所述图像块在所述基于模板匹配的几何划分模式下被划分为第一分区和第二分区。
可选地，包括以下至少一项：所述基于模板匹配的几何划分模式的类型包括：基于模板匹配的帧内预测和/或基于模板匹配的帧间预测；所述第一分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测；所述第二分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测。
在一个实施例中，所述目标预测模式包括基于模板匹配的几何划分模式，处理器110，用于：通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板；根据所述相似模板确定所述图像块的预测结果。
在一个实施例中，处理器110，用于：根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板。
可选地，所述预定搜索范围基于参考块对应的相似模板和/或参考块的预测候选块的位置确定。
在一个实施例中，处理器110，还用于：根据所述参考块和所述图像块之间的位置关系、所述参考块对应的相似模板的位置，确定所述预定搜索范围；和/或，根据所述参考块和所述图像块之间的位置关系、所述参考块的预测候选块的位置，确定所述预定搜索范围。
在一个实施例中，处理器110，用于：若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则从所述图像块所在的图像中确定当前模板对应的至少一个相似模板；和/或，若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板。
在一个实施例中，处理器110，用于：按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板；根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
在一个实施例中，处理器110，用于：从所述图像块所在的图像对应的第一参考图像中确定当前模板对应的第一相似模板，和/或，从所述图像块所在的图像对应的第二参考图像中确定当前模板对应的第二相似模板，将所述第一相似模板和/或所述第二相似模板确定为所述当前模板对应的至少一个相似模板；根据至少一个初始运动矢量从所述图像块所在的图像对应的参考图像中确定至少一个初始相似模板，在包括任一个所述初始相似模板的预定范围内搜索与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
可选地，包括以下至少一项：处理器110，用于：在对在所述图像块所在的图像中的重建图像区域进行搜索的过程中，当出现预设数量个满足匹配条件的目标区域，则将各个满足匹配条件的目标区域确定为与当前模板匹配的相似模板；和/或，当所述重建图像区域搜索完毕之后，将满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板。
可选地，处理器110，用于：确定所匹配的相似模板与所述当前模板之间的模板代价，将满足预设模板条件的模板代价对应的所匹配的相似模板确定为所述当前模板对应的至少一个相似模板。
在一个实施例中，处理器110，用于：根据对应分区的至少一个相似模板确定该分区的至少一个预测候选块；根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果。
在一个实施例中，处理器110，用于：获取所述图像块的初始当前模板；将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板。
在一个实施例中，处理器110，用于以下至少一项：通过第一标志确定所述图像块的目标预测模式；将最小率失真代价对应的预测模式确定为所述图像块的目标预测模式。
可选地，若所述目标预测模式是基于模板匹配的几何划分模式，则在确定所述预测结果的过程中，利用第一标志指示所述图像块使用所述基于模板匹配的几何划分模式，以及利用至少一第二标志指示所述图像块的第一分区和/或第二分区各自使用的预测处理。
在一个实施例中，处理器110，用于：根据所述目标预测模式和所述预设策略，确定所述图像块的至少一个分区中的每一个分区对应的加权后的预测候选块，以用于确定所述图像块的预测结果。
在一个实施例中，若所述目标预测模式为基于模板匹配的几何划分模式，则处理器110，用于：确定所述图像块的初始当前模板，以用于通过所述初始当前模板确定所述图像块的预测结果；可选地，所述初始当前模板包括分区模板区域和非分区模板区域。
可选地，分区模板区域包括第一分区对应的第一当前模板以及第二分区对应的第二当前模板。
可选地，初始当前模板包括与所述图像块的上边界相邻的区域、与所述图像块的左边界相邻的区域、与所述图像块左上方的相邻区域中的至少一个区域；所述初始当前模板包括所述第一分区对应的当前模板和所述第二分区对应的当前模板。
可选地，所述第一分区对应的当前模板的横坐标范围与所述第二分区对应的当前模板的横坐标范围不同；或者，所述第一分区对应的当前模板的纵坐标范围与所述第二分区对应的当前模板的纵坐标范围不同；或者，所述第一分区对应的当前模板的长和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的长不相等；或者，所述第一分区对应的当前模板的宽和/或所述第二分区对应的当前模板的长与所述图像块和/或所述图像块的预测候选块的宽不相等。
可选地，所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板包括至少一边界，所述至少一边界位于所述图像块使用的分割线所在的直线上；或者，
所述初始当前模板包括所述图像块的第一分区对应的当前模板、所述图像块的第二分区对应的当前模板、以及非分区模板区域，其中所述图像块使用的分割线所在的直线经过所述非分区模板区域；或者所述非分区模板区域与所述图像块的 第一分区对应的当前模板与所述图像块的第二分区对应的当前模板中的至少一个相邻。
可选地，所述当前模板的类型包括斜边模板和/或非斜边模板，所述至少一个相似模板的类型和所述当前模板的类型相同；当所述当前模板的类型为所述斜边模板时，所述斜边模板的斜边位置和斜率根据划分所述图像块的分割线设置；当所述当前模板的类型为所述非斜边模板时，所述非斜边模板的位置根据所述分割线所在直线经过所述初始当前模板的区域进行设置。
应当理解，本申请实施例中所描述的移动终端可执行上述任一实施例的方法描述，也可执行上述所对应实施例中对该图像处理装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例还提供一种计算机可读存储介质，存储介质上存储有图像处理程序，图像处理程序被处理器执行时实现上述任一实施例中的图像处理方法的步骤。
在本申请实施例提供的智能终端和计算机可读存储介质的实施例中，可以包含任一上述图像处理方法实施例的全部技术特征，说明书拓展和解释内容与上述方法的各实施例基本相同，在此不再做赘述。
图10是本申请实施例示出的一种控制器的硬件结构示意图。该控制器140包括：存储器1401和处理器1402，存储器1401用于存储程序指令，处理器1402用于调用存储器1401中的程序指令执行上述方法实施例中控制器所执行的步骤，其实现原理以及有益效果类似，此处不再进行赘述。
可选地，上述控制器还包括通信接口1403，该通信接口1403可以通过总线1404与处理器1402连接。处理器1402可以控制通信接口1403来实现控制器140的接收和发送的功能。
图11是本申请实施例示出的一种网络节点的硬件结构示意图。该网络节点150包括：存储器1501和处理器1502，存储器1501用于存储程序指令，处理器1502用于调用存储器1501中的程序指令执行上述方法实施例中首节点所执行的步骤，其实现原理以及有益效果类似，此处不再进行赘述。
可选地，上述网络节点还包括通信接口1503，该通信接口1503可以通过总线1504与处理器1502连接。处理器1502可以控制通信接口1503来实现网络节点150的接收和发送的功能。
本申请实施例还提供一种计算机程序产品，计算机程序产品包括计算机程序代码，当计算机程序代码在计算机上运行时，使得计算机执行如上各种可能的实施方式中的方法。
本申请实施例还提供一种芯片，包括存储器和处理器，存储器用于存储计算机程序，处理器用于从存储器中调用并运行计算机程序，使得安装有芯片的设备执行如上各种可能的实施方式中的方法。
可以理解，上述场景仅是作为示例，并不构成对于本申请实施例提供的技术方案的应用场景的限定，本申请的技术方案还可应用于其他场景。例如，本领域普通技术人员可知，随着系统架构的演变和新业务场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
本申请实施例方法中的步骤可以根据实际需要进行顺序调整、合并和删减。
本申请实施例设备中的单元可以根据实际需要进行合并、划分和删减。
在本申请中，对于相同或相似的术语概念、技术方案和/或应用场景描述，一般只在第一次出现时进行详细描述，后面再重复出现时，为了简洁，一般未再重复阐述，在理解本申请技术方案等内容时，对于在后未详细描述的相同或相似的术语概念、技术方案和/或应用场景描述等，可以参考其之前的相关详细描述。
在本申请中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本申请技术方案的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本申请记载的范围。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，被控终端，或者网络设备等)执行本申请每个实施例的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络，或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、存储盘、磁带)、光介质(例如，DVD)，或者半导体介质(例如固态存储盘Solid State Disk(SSD))等。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (17)

  1. 一种图像处理方法，其中，包括以下步骤：
    S1：获取或确定图像块的目标预测模式；
    S2：根据所述目标预测模式确定所述图像块的预测结果。
  2. 如权利要求1所述的方法，其中，所述图像块被划分为第一分区和第二分区，所述目标预测模式包括第一帧内预测模式、第一帧间预测模式、第二帧内预测模式、第二帧间预测模式中的至少一个，以及
    所述第一分区采用所述第一帧内预测模式或者所述第一帧间预测模式，或者，所述第二分区采用所述第二帧内预测模式或者所述第二帧间预测模式。
  3. 如权利要求2所述的方法，其中，所述第一帧间预测模式为单向预测模式或者双向预测模式，或者，所述第二帧间预测模式为单向预测模式或者双向预测模式。
  4. 如权利要求1所述的方法，其中，所述目标预测模式包括基于模板匹配的几何划分模式，所述图像块在所述基于模板匹配的几何划分模式下被划分为第一分区和第二分区。
  5. 如权利要求4所述的方法，其中，包括以下至少一项：
    所述基于模板匹配的几何划分模式的类型包括：基于模板匹配的帧内预测和/或基于模板匹配的帧间预测；
    所述第一分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测；
    所述第二分区采用所述基于模板匹配的帧内预测或者所述基于模板匹配的帧间预测。
  6. 如权利要求1至5中任一项所述的方法，其中，所述目标预测模式包括基于模板匹配的几何划分模式，所述S2步骤，包括：
    通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板；
    根据所述相似模板确定所述图像块的预测结果。
  7. 如权利要求6所述的方法，其中，所述通过预设策略确定所述图像块中各个分区对应当前模板中的至少一个相似模板，包括：
    根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板。
  8. 如权利要求7所述的方法，其中，所述预定搜索范围基于参考块对应的相似模板和/或参考块的预测候选块的位置确定。
  9. 如权利要求8所述的方法，其中，所述方法还包括：
    根据所述参考块和所述图像块之间的位置关系、所述参考块对应的相似模板的位置，确定所述预定搜索范围；和/或，
    根据所述参考块和所述图像块之间的位置关系、所述参考块的预测候选块的位置，确定所述预定搜索范围。
  10. 如权利要求7至9中任一项所述的方法，其中，所述根据基于模板匹配的几何划分模式的类型从预定搜索范围内确定当前模板对应的至少一个相似模板，包括：
    若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧内预测，则从所述图像块所在的图像中确定当前模板对应的至少一个相似模板；和/或，
    若所述基于模板匹配的几何划分模式的类型为基于模板匹配的帧间预测，则从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板。
  11. 如权利要求10所述的方法，其中，包括以下至少一项：
    所述从所述图像块所在的图像中确定当前模板对应的至少一个相似模板，包括：按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板；
    所述从所述图像块所在的图像对应的参考图像中确定当前模板对应的至少一个相似模板，包括：从所述图像块所在的图像对应的第一参考图像中确定当前模板对应的第一相似模板，和/或，从所述图像块所在的图像对应的第二参考图像中确定当前模板对应的第二相似模板，将所述第一相似模板和/或所述第二相似模板确定为所述当前模板对应的至少一个相似模板；和/或，根据至少一个初始运动矢量从所述图像块所在的图像对应的参考图像中确定至少一个初始相似模板，在包括任一个所述初始相似模板的预定范围内搜索与当前模板匹配的相似模板，根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板。
  12. 如权利要求11所述的方法，其中，包括以下至少一项：
    所述按照预设搜索规则从所述图像块所在的图像中的重建图像区域中，确定与当前模板匹配的相似模板，包括：在对在所述图像块所在的图像中的重建图像区域进行搜索的过程中，当出现预设数量个满足匹配条件的目标区域，则将各个满足匹配条件的目标区域确定为与当前模板匹配的相似模板；和/或，当所述重建图像区域搜索完毕之后，将满足匹配条件的预设数量个目标区域确定为与当前模板匹配的相似模板；
    所述根据所匹配的相似模板确定所述当前模板对应的至少一个相似模板，包括：确定所匹配的相似模板与所述当前模板 之间的模板代价，将满足预设模板条件的模板代价对应的所匹配的相似模板确定为所述当前模板对应的至少一个相似模板。
  13. 如权利要求6所述的方法，其中，所述根据所述相似模板确定所述图像块的预测结果，包括：
    根据对应分区的至少一个相似模板确定该分区的至少一个预测候选块；
    根据所述图像块中各个分区的所述至少一个预测候选块确定所述图像块的预测结果。
  14. 如权利要求6所述的方法，其中，所述方法还包括：
    获取所述图像块的初始当前模板；
    将所述初始当前模板划分为所述图像块的第一分区对应的当前模板和/或所述图像块的第二分区对应的当前模板。
  15. 如权利要求1所述的方法，其中，所述S1步骤，包括以下至少一项：
    通过第一标志确定所述图像块的目标预测模式；
    将最小率失真代价对应的预测模式确定为所述图像块的目标预测模式。
  16. 一种智能终端，其中，包括：存储器、处理器，其中，所述存储器上存储有图像处理程序，所述图像处理程序被所述处理器执行时实现如权利要求1所述的图像处理方法的步骤。
  17. 一种计算机可读存储介质，其中，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1所述的图像处理方法的步骤。
PCT/CN2023/095627 2022-07-15 2023-05-22 图像处理方法、智能终端及存储介质 WO2024012054A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210832482.1A CN115002463B (zh) 2022-07-15 2022-07-15 图像处理方法、智能终端及存储介质
CN202210832482.1 2022-07-15

Publications (1)

Publication Number Publication Date
WO2024012054A1 true WO2024012054A1 (zh) 2024-01-18

Family

ID=83022180

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/095627 WO2024012054A1 (zh) 2022-07-15 2023-05-22 图像处理方法、智能终端及存储介质

Country Status (2)

Country Link
CN (1) CN115002463B (zh)
WO (1) WO2024012054A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115002463B (zh) * 2022-07-15 2023-01-13 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质
CN116909718A (zh) * 2023-09-13 2023-10-20 北京谷器数据科技有限公司 一种基于线程等待和异步报工的并行计算方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111246212A (zh) * 2020-02-21 2020-06-05 北京大学 一种基于编解码端的几何划分模式预测方法、装置、存储介质及终端
CN112291558A (zh) * 2019-07-23 2021-01-29 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
US20210084329A1 (en) * 2018-06-01 2021-03-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Video codec using template matching prediction
WO2021196235A1 (zh) * 2020-04-03 2021-10-07 Oppo广东移动通信有限公司 帧间预测方法、编码器、解码器及存储介质
CN114598880A (zh) * 2022-05-07 2022-06-07 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质
KR20220089685A (ko) * 2020-12-21 2022-06-28 한국전자통신연구원 분할을 사용하는 영상 부호화/복호화를 위한 방법, 장치 및 기록 매체
CN115002463A (zh) * 2022-07-15 2022-09-02 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101215152B1 (ko) * 2011-04-21 2012-12-24 한양대학교 산학협력단 인루프 필터링을 적용한 예측 방법을 이용한 영상 부호화/복호화 방법 및 장치
KR102319384B1 (ko) * 2014-03-31 2021-10-29 인텔렉추얼디스커버리 주식회사 템플릿 매칭 기반의 화면 내 픽쳐 부호화 및 복호화 방법 및 장치
WO2021015581A1 (ko) * 2019-07-23 2021-01-28 한국전자통신연구원 기하학적 분할을 사용하는 영상 부호화/복호화를 위한 방법, 장치 및 기록 매체
MX2023003383A (es) * 2020-09-28 2023-03-31 Interdigital Ce Patent Holdings Sas Prediccion de coincidencia de plantilla para codificacion versatil de video.
CN113596475A (zh) * 2021-06-24 2021-11-02 浙江大华技术股份有限公司 图像/视频编码方法、装置、系统及计算机可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210084329A1 (en) * 2018-06-01 2021-03-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Video codec using template matching prediction
CN112291558A (zh) * 2019-07-23 2021-01-29 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
CN111246212A (zh) * 2020-02-21 2020-06-05 北京大学 一种基于编解码端的几何划分模式预测方法、装置、存储介质及终端
WO2021196235A1 (zh) * 2020-04-03 2021-10-07 Oppo广东移动通信有限公司 帧间预测方法、编码器、解码器及存储介质
KR20220089685A (ko) * 2020-12-21 2022-06-28 한국전자통신연구원 분할을 사용하는 영상 부호화/복호화를 위한 방법, 장치 및 기록 매체
CN114598880A (zh) * 2022-05-07 2022-06-07 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质
CN115002463A (zh) * 2022-07-15 2022-09-02 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质

Also Published As

Publication number Publication date
CN115002463B (zh) 2023-01-13
CN115002463A (zh) 2022-09-02

Similar Documents

Publication Publication Date Title
WO2024012054A1 (zh) 图像处理方法、智能终端及存储介质
WO2023216866A1 (zh) 图像处理方法、智能终端及存储介质
CN112822491B (zh) 一种图像数据的编码、解码方法及装置
US10986332B2 (en) Prediction mode selection method, video encoding device, and storage medium
WO2023185351A1 (zh) 图像处理方法、智能终端及存储介质
WO2019169997A1 (zh) 视频运动估计方法、装置、终端及存储介质
CN109934769B (zh) 屏幕长截图的方法、终端及存储介质
EP4277270A1 (en) Screen content processing method and apparatus, and device
CN115988206B (zh) 图像处理方法、处理设备及存储介质
WO2019001015A1 (zh) 一种图像数据的编码、解码方法及装置
US10887586B2 (en) Picture encoding method and terminal
CN116456102B (zh) 图像处理方法、处理设备及存储介质
US10827198B2 (en) Motion estimation method, apparatus, and storage medium
CN116668704A (zh) 处理方法、处理设备及存储介质
CN109669819B (zh) 网页开启方法、装置、移动终端以及计算机可读存储介质
WO2023108444A1 (zh) 图像处理方法、智能终端及存储介质
CN110209449A (zh) 一种游戏中光标定位方法和装置
WO2024087604A1 (zh) 图像处理方法、智能终端及存储介质
CN116847088B (zh) 图像处理方法、处理设备及存储介质
CN114125151B (zh) 图像处理方法、移动终端及存储介质
CN110276181B (zh) 信息处理方法、电子设备及计算机存储介质
WO2024098873A1 (zh) 处理方法、处理设备及存储介质
WO2024040962A1 (zh) 无线感知方法、通信设备及存储介质
WO2023019567A1 (zh) 图像处理方法、移动终端及存储介质
CN110213593B (zh) 一种运动矢量的确定方法、编码压缩方法和相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23838548

Country of ref document: EP

Kind code of ref document: A1