CN115379214A

CN115379214A - Image processing method, intelligent terminal and storage medium

Info

Publication number: CN115379214A
Application number: CN202211314785.0A
Authority: CN
Inventors: 刘雨田
Original assignee: Shenzhen Transsion Holdings Co Ltd
Current assignee: Shenzhen Transsion Holdings Co Ltd
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2022-11-22
Anticipated expiration: 2042-10-26
Also published as: CN115379214B; WO2024087604A1

Abstract

The application provides an image processing method, an intelligent terminal and a storage medium, wherein the image processing method comprises the following steps: acquiring or determining the size of a fusion area of a block to be processed; determining a prediction result of the block to be processed according to the size of the fusion area; and carrying out image coding and/or decoding processing according to the prediction result. According to the method and the device, the size of the fusion region with high possibility of being used is directly selected in the process of predicting the image block, so that the prediction result of the block to be processed is determined based on the size of the fusion region and is used for image coding and/or decoding processing, the image coding time is effectively reduced, the overall coding and decoding efficiency of the video is ensured, and the user experience is improved.

Description

Image processing method, intelligent terminal and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, an intelligent terminal, and a storage medium.

Background

The existing multifunctional video codec standard (h.266) proposes a GPM (Geometric partitioning mode) mode. The GPM mode can divide the boundary part of a moving object in a video sequence more finely, so that the coding and decoding performance is improved to a certain extent.

In the course of conceiving and implementing the present application, the inventors found that at least the existing standards had the following problems: the GPM mode adopted today is fixed for the width of the fusion region, but there are many situations in the boundary width of a moving object in a video sequence, which results in that the GPM mode adopts the fixed fusion region width to predict the image blocks in the video image, and is not all suitable in practice. Therefore, it is necessary to provide an innovative image processing method to improve the prediction accuracy of the GPM mode on the video image and the encoding/decoding performance of the image.

The foregoing description is provided for general background information and is not admitted to be prior art.

Disclosure of Invention

In view of the foregoing technical problems, the present application provides an image processing method, an intelligent terminal, and a storage medium, so that in a process of predicting an image block, a size of a fusion region that is likely to be used may be directly selected, thereby effectively reducing an image encoding time to ensure overall encoding and decoding efficiency of a video.

The application provides an image processing method which can be applied to an intelligent terminal and comprises the following steps:

s10: acquiring or determining the size of a fusion area of a block to be processed;

s20: determining a prediction result of the block to be processed according to the size of the fusion area;

s30: and carrying out image coding or decoding processing on the block to be processed according to the prediction result.

The present application also provides an image processing apparatus including:

the first determining module is used for acquiring or determining the size of a fusion area of a block to be processed;

a second determining module, configured to determine a prediction result of the block to be processed according to the size of the fusion region;

and the processing module is used for carrying out image coding or decoding processing on the block to be processed according to the prediction result.

The application also provides an intelligent terminal, including: the image processing method comprises a memory and a processor, wherein the memory stores an image processing program, and the image processing program realizes the steps of any one of the image processing methods when being executed by the processor.

The present application also provides a storage medium storing a computer program which, when executed by a processor, implements the steps of the image processing method as described in any one of the above.

As described above, the image processing method of the present application includes the steps of: s10: acquiring or determining the size of a fusion area of a block to be processed; s20: determining a prediction result of the block to be processed according to the size of the fusion area; s30: and carrying out image coding or decoding processing on the block to be processed according to the prediction result. By the technical scheme, the size of the fusion region with high possibility of being used can be directly selected in the process of predicting the image block, and the prediction result of the block to be processed is determined based on the size of the fusion region and is used for encoding or decoding the image of the block to be processed, so that the prediction precision of the image is improved, the image encoding time is effectively reduced, the overall encoding and decoding performance of the video image is improved, and the user experience is further improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic hardware structure diagram of a mobile terminal implementing various embodiments of the present application;

fig. 2 is a diagram illustrating a communication network system architecture according to an embodiment of the present application;

FIG. 3a is a schematic view of an angular distribution provided by an embodiment of the present application;

FIG. 3b is a schematic diagram of various offsets at an angle φ i according to an embodiment of the present application;

FIG. 3c is a schematic diagram of a partition mode provided in an embodiment of the present application;

fig. 4 is a flowchart illustrating an image processing method according to the first embodiment;

FIG. 5 is a diagram of constructing a most likely fused size candidate list provided in accordance with the first embodiment;

fig. 6 is a flowchart illustrating an image processing method according to a second embodiment;

fig. 7 is a schematic diagram of a partition boundary based on the Hessian paradigm according to an embodiment of the present application;

FIG. 8 is a diagram illustrating an intra template matching search for use in GPM mode according to an embodiment of the present disclosure;

FIG. 9 is a diagram illustrating an interframe template matching search used in GPM mode according to an embodiment of the present disclosure;

fig. 10 is a flowchart illustrating an image processing method according to a third embodiment;

FIG. 11a is a schematic diagram of a comparison area of a current template and the current template provided in an embodiment of the present application;

FIG. 11b is a schematic diagram of a similar template comparison area and a similar template provided in an embodiment of the present application;

FIG. 11c is a schematic view of an overlap region provided by an embodiment of the present application;

fig. 12a, fig. 12b and fig. 12c are distribution diagrams of values of the weight W2 of the comparison region Bc1 of a similar template according to an embodiment of the present application;

fig. 12d, fig. 12e and fig. 12f are value distribution diagrams of the weight W2 of the comparison region Bc1 of the similar template provided in the embodiment of the present application;

FIG. 13 is a schematic diagram of a sample passed by a straight line where a GPM partition line is located according to an embodiment of the present application;

fig. 14 is a schematic diagram of an image processing apparatus according to an embodiment of the present application.

The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings. With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the recitation of a claim "comprising a" 8230a "\8230means" does not exclude the presence of additional identical elements in the process, method, article or apparatus in which the element is incorporated, and further, similarly named components, features, elements in different embodiments of the application may have the same meaning or may have different meanings, the specific meaning of which should be determined by its interpretation in the specific embodiment or by further combination with the context of the specific embodiment.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope herein. The word "if" as used herein may be interpreted as "at" \8230; "or" when 8230; \8230; "or" in response to a determination ", depending on the context. Also, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, steps, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, steps, operations, elements, components, species, and/or groups thereof. As used herein, the terms "or," "and/or," "including at least one of the following," and the like, are to be construed as inclusive or meaning any one or any combination. For example, "includes at least one of: A. b, C "means" any of the following: a; b; c; a and B; a and C; b and C; a and B and C ", again for example," a, B or C "or" a, B and/or C "means" any one of the following: a; b; c; a and B; a and C; b and C; a and B and C'. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

It should be understood that, although the steps in the flowcharts in the embodiments of the present application are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, in different orders, and may be performed alternately or at least partially with respect to other steps or sub-steps of other steps.

The words "if", as used herein may be interpreted as "at \8230; \8230whenor" when 8230; \8230when or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.

It should be noted that step numbers such as S10 and S20 are used herein for the purpose of more clearly and briefly describing the corresponding contents, and do not constitute a substantial limitation on the sequence, and those skilled in the art may perform S20 first and then S10 in the specific implementation, but these should be within the protection scope of the present application.

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of description of the present application, and have no specific meaning in themselves. Thus, "module", "component" or "unit" may be used mixedly.

The intelligent terminal may be implemented in various forms. For example, the smart terminal described in the present application may include smart terminals such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and fixed terminals such as a Digital TV, a desktop computer, and the like.

The following description will be given taking a mobile terminal as an example, and it will be understood by those skilled in the art that the configuration according to the embodiment of the present application can be applied to a fixed type terminal in addition to elements particularly used for mobile purposes.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency) unit 101, wiFi module 102, audio output unit 103, a/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111. Those skilled in the art will appreciate that the mobile terminal architecture shown in fig. 1 is not intended to be limiting of mobile terminals, which may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile terminal in detail with reference to fig. 1:

the radio frequency unit 101 may be configured to receive and transmit signals during information transmission and reception or during a call, and specifically, receive downlink information of a base station and then process the downlink information to the processor 110; in addition, the uplink data is transmitted to the base station. Typically, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can also communicate with a network and other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000 ), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division duplex-Long Term Evolution), TDD-LTE (Time Division duplex-Long Term Evolution ), 5G (Global System for Mobile communications, or the like).

WiFi belongs to a short-distance wireless transmission technology, and the mobile terminal can help a user to receive and send emails, browse webpages, access streaming media and the like through the WiFi module 102, and provides wireless broadband internet access for the user. Although fig. 1 shows the WiFi module 102, it is understood that it does not belong to the essential constitution of the mobile terminal, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is used to receive audio or video signals. The a/V input Unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042, and the Graphics processor 1041 processes image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphic processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 may receive sounds (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, or the like, and may be capable of processing such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 101 in case of the phone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor that may adjust the brightness of the display panel 1061 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 1061 and/or the backlight when the mobile terminal 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the gesture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.

The display unit 106 is used to display information input by a user or information provided to the user. The Display unit 106 may include a Display panel 1061, and the Display panel 1061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal. Alternatively, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, can collect touch operations of a user (e.g., operations of a user on the touch panel 1071 or near the touch panel 1071 using a finger, a stylus, or any other suitable object or accessory) thereon or nearby and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. Optionally, the touch detection device detects a touch orientation of a user, detects a signal caused by a touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like, and are not limited thereto.

Alternatively, the touch panel 1071 may cover the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited herein.

The interface unit 108 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and external devices.

The memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a program storage area and a data storage area, and optionally, the program storage area may store an operating system, an application program (such as a sound playing function, an image playing function, and the like) required by at least one function, and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 109 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by operating or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, optionally the application processor primarily handles operating systems, user interfaces, application programs, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The mobile terminal 100 may further include a power supply 111 (e.g., a battery) for supplying power to various components, and preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described in detail herein.

In order to facilitate understanding of the embodiments of the present application, a communication network system on which the mobile terminal of the present application is based is described below.

Referring to fig. 2, fig. 2 is an architecture diagram of a communication Network system according to an embodiment of the present disclosure, where the communication Network system is an LTE system of a universal mobile telecommunications technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an epc (Evolved Packet Core) 203, and an IP service 204 of an operator, which are in communication connection in sequence.

Optionally, the UE201 may be the terminal 100 described above, and details are not described here.

The E-UTRAN202 includes eNodeB2021 and other eNodeBs 2022, among others. Alternatively, the eNodeB2021 may be connected with other enodebs 2022 through a backhaul (e.g., X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide the UE201 access to the EPC 203.

The EPC203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving gateway) 2034, a PGW (PDN gateway) 2035, and a PCRF (Policy and Charging Rules Function) 2036, and the like. Optionally, the MME2031 is a control node that handles signaling between the UE201 and the EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location register (not shown) and holds subscriber specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034, PGW2035 may provide IP address assignment for UE201 and other functions, and PCRF2036 is a policy and charging control policy decision point for traffic data flow and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).

IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem), or other IP services, among others.

Although the LTE system is described as an example, it should be understood by those skilled in the art that the present application is not limited to the LTE system, but may also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G, and future new network systems (e.g. 6G), and the like.

Based on the above mobile terminal hardware structure and communication network system, various embodiments of the present application are provided.

For the sake of understanding, the following first explains the terms of art to which the embodiments of the present application may be related.

1. Inter prediction mode

In the process of coding an image, predicting an image block is an indispensable step, and a prediction block is obtained by predicting the image block, so that a residual block with smaller energy is constructed, and transmission bits are reduced. The prediction of the image block may be implemented by some preset prediction modes, which may include an inter prediction mode and an intra prediction mode. The inter prediction mode is a prediction mode having higher coding efficiency, which removes temporal redundancy using correlation between pixels, compared to the intra prediction mode.

2. Geometric Partitioning Mode (GPM)

The GPM mode is a prediction mode adopted by the next generation Video compression standard (VVC) for the boundary portion of a moving object in an image. Compared with other prediction modes, the GPM mode can divide the boundary of a moving object in an image more finely, and divides the edge coding unit (Code unit) of the moving object into non-rectangular sub coding units for unidirectional prediction by attaching a dividing line to the boundary of the moving object, so that the prediction value of the whole coding unit is obtained.

FIG. 3a is a schematic view of an angle distribution, which can be found that the angle corresponding to each dividing line is different, and the angle is labeled as φ (i =1 i ~ 24). Fig. 3a does not fully illustrate all angles. There are a maximum of 4 offsets at each angle as shown in figure 3 b. FIG. 3b is a schematic diagram of 4 offsets at an angle φ i according to an embodiment of the present application. Wherein the offsets are marked as ρ (j =0 j ~ 3). Alternatively, 64 partition modes can be combined, as shown in fig. 3 c. Fig. 3c is a schematic diagram of a partition mode according to an embodiment of the present application. The division mode comprises at least one division mode formed by different angle and offset combinations. It can be seen that the positions of the dividing lines are different at different angles and different offsets, and the number of dividing lines that can be used for the image block is also different. Any one of all the division lines is one implementation way of applying the GPM mode to the image block. Optionally, the offset ρ j and the angle φ i may be determined from the GPM mode index.

First embodiment

Referring to fig. 4, fig. 4 is a flowchart illustrating an image processing method according to a first embodiment. In this embodiment, an execution subject of the image processing method may be a computer device or a cluster formed by a plurality of computer devices, and the computer device may be an intelligent terminal (such as the mobile terminal 100) or a server, where the execution subject in this embodiment is an intelligent terminal for example.

In the process of video encoding or decoding, an encoder at an encoding end generally divides an input video frame into a plurality of image blocks for processing, each image block may be subtracted from a prediction block obtained through prediction in a prediction mode to obtain a residual block, and then a series of processing is performed on the residual block to obtain an encoded bitstream. Then, at the decoding end, after the decoder receives the bit stream, the decoder can obtain the prediction mode parameters by analyzing the bit stream. Furthermore, the inverse transform unit and the inverse quantization unit of the decoder perform inverse transform and inverse quantization processing on the transform coefficient to obtain a residual block. In addition, the decoding unit of the decoder parses and decodes the encoded bitstream to obtain the prediction parameters and the related side information. Next, the prediction processing unit of the decoder performs prediction processing using the prediction parameters, thereby determining a prediction block corresponding to the residual block. In this way, the decoder can obtain a reconstructed block by adding the obtained residual block and the corresponding prediction block. Meanwhile, the loop filtering unit of the decoder performs loop filtering processing on the reconstruction block to reduce distortion and improve video quality. Thus, the reconstructed blocks subjected to the loop filter processing are further combined into a decoded image, stored in a decoded image buffer, or output as a decoded video signal.

The image processing method provided by the embodiment of the application can be applied to a scene for predicting the image block in the video coding process, and can also be applied to a scene for predicting the image block to be decoded in the video decoding process to obtain a prediction block.

The image processing method provided by the embodiment of the application comprises the following steps:

optionally, in this embodiment of the present application, the block to be processed may be an image block currently being encoded or decoded in an input Video image (i.e., a Video frame), and may be referred to as a current block or a current image block for short, and under the h.265/High Efficiency Video Coding (HEVC) standard, the block to be processed may be a Coding Tree Unit (CTU) or a Coding Unit (CU) in the input Video image, which is not limited in this embodiment of the present application.

Alternatively, the intelligent terminal as an encoder performs a prediction process on each image block by using a temporal and/or spatial correlation between video images after receiving the video images from the video source, that is, after dividing the video images into a plurality of image blocks. When the intelligent terminal adopts the GPM mode to carry out prediction processing on the current image block, firstly, the size of a fusion area of a block to be processed which needs prediction processing currently is obtained or determined by using a partition mode in the GPM mode.

Optionally, the prediction processing performed by the intelligent terminal for the block to be processed may include intra-frame prediction processing and/or inter-frame prediction processing, and the intra-frame prediction processing and/or the inter-frame prediction processing each include a plurality of prediction modes respectively. The intelligent terminal may specifically determine the prediction mode that is finally adopted currently for each block to be processed in the plurality of image blocks using, for example, a rate-distortion optimization manner. For example, the intelligent terminal determines the minimum rate-distortion cost from the rate-distortion costs respectively corresponding to the plurality of prediction modes by calculating the rate-distortion cost corresponding to each prediction mode. And taking the prediction mode corresponding to the minimum rate-distortion cost as the prediction mode of the final sampling of each block to be processed in the plurality of image blocks at present. Thus, when the rate-distortion cost corresponding to the GPM mode among the multiple prediction modes is calculated to be the minimum rate-distortion cost, the intelligent terminal adopts the GPM mode to predict the image block.

Alternatively, after the intelligent terminal receives the bit stream that has been encoded by the encoder as a decoder, a decoding unit of the decoder parses and decodes the bit stream to obtain the prediction parameters. Meanwhile, the inverse transformation unit and the inverse quantization unit of the decoder perform inverse transformation and inverse quantization processing on the transformation coefficient to obtain a residual block. Next, the prediction processing unit of the decoder may use the residual block as a block to be processed that currently needs to be decoded to perform prediction processing using the prediction parameters, thereby determining a prediction block corresponding to the residual block. Optionally, when the decoder adopts the same GPM mode as that used by the encoder, the decoder may obtain or determine the size of the fusion region of the block to be processed, which needs to be subjected to prediction processing currently, by using the prediction parameters obtained by parsing the bitstream.

Optionally, after the intelligent terminal is used as a decoder to obtain the prediction parameters by parsing and decoding the bitstream through the decoding unit, if the prediction parameters indicate that the corresponding prediction mode is the GPM mode, the intelligent terminal may directly use the GPM mode as the prediction mode for performing prediction processing on the obtained residual block.

Optionally, the step S10 includes:

and acquiring or determining the size of the fusion area according to a preset image block template.

Optionally, in the process that the intelligent terminal performs prediction processing on the block to be processed in the GPM mode, the intelligent terminal first determines or generates a fusion size candidate list, and then the terminal device may obtain or determine, according to a preset image block template, a fusion region size of the block to be processed when prediction processing is performed on the current block to be processed, from among one or more fusion size candidates included in the fusion size candidate list.

Optionally, the size of the fusion area is determined according to the size of the block size to be processed. For example, if the size of the block to be processed is a1 × a1, a first blend region size is selected, and if the size of the block to be processed is a2 × a2, a second blend region size is selected. Optionally, a1 is not equal to a2.

Optionally, the preset image block template is an image block template in a GPM mode, which is a template-based prediction mode adopted for performing prediction processing on a current block to be processed. Optionally, the fused size candidate list includes at least one most likely fused size candidate.

Exemplarily, please refer to fig. 5, wherein fig. 5 is a schematic diagram for constructing a fusion size candidate list provided in an embodiment of the present application. The intelligent terminal as an encoder can select one or more most probable fusion size candidates from a plurality of fusion sizes in advance through a most probable fusion size candidate determining module and a fusion size candidate list constructing module configured by the intelligent terminal, and then construct a fusion size candidate list by using the one or more most probable fusion size candidates. That is, it is assumed that, in the process of performing prediction processing on an image block by using a GPM mode by an intelligent terminal, a fusion size existing in the GPM mode is τ/4, τ/2, τ,2 τ,4 τ,8 τ.. Once, optionally, τ is one sample in the image, when the fusion size is τ/2, a region corresponding to 1/2 sampling points on the left and right of a GPM partition line is a fusion region, when the fusion size is τ, a region corresponding to 1 sampling point on the left and right of the GPM partition line is a fusion region, and when the fusion size is 2 τ, a region corresponding to 2 sampling points on the left and right of the GPM partition line is a fusion region. It should be understood that in other embodiments, τ may be other unit lengths.

Optionally, if the intelligent terminal determines that the number of the most probable fusion size candidates in the fusion size candidate list that needs to be currently constructed is 3, the intelligent terminal may select τ/4, τ/2, τ from τ/4, τ/2, 2 τ,4 τ,8 τ, or select τ,2 τ,4 τ, or select τ/4, τ,4 τ as the three most probable fusion size candidates. Optionally, the intelligent terminal may also determine a plurality of most probable fusion size candidates according to other strategies. Therefore, after the intelligent terminal selects the 3 most probable fusion size candidates, a fusion size candidate list can be constructed. For example, the intelligent terminal sets a fusion size candidate list [ x ], wherein x =0 to 2. If τ/4, τ/2, τ is the most probable fusion size candidate, then the fusion size candidate list [0] = τ/4, the fusion size candidate list [1] = τ/2, and the fusion size candidate list [2] = τ may be set.

Optionally, in addition to obtaining or determining the fusion region size of the block to be processed from the fusion size candidate list through the preset image block template, the intelligent terminal, as an encoder, may further sequentially calculate and determine, for each most probable fusion size candidate, a rate-distortion cost corresponding to each most probable fusion size to which the most probable fusion size is applied after obtaining or determining a plurality of most probable fusion size candidates in the fusion size candidate list, so as to determine a minimum rate-distortion cost among a plurality of rate-distortion costs, so that the intelligent terminal may determine the most probable fusion size candidate corresponding to the minimum rate-distortion cost as the fusion region size finally adopted by the block to be processed when performing prediction processing on the current block to be processed.

Optionally, after the intelligent terminal is used as an encoder to obtain or determine the final adopted fusion region size of the block to be processed from the fusion size candidate list, the fusion region size may be encoded and then packed into a bitstream. Therefore, at a decoding end, the intelligent terminal can directly analyze the bit stream to obtain the size of a fusion area finally adopted by the block to be processed, and the weight corresponding to the prediction block is determined according to the size of the fusion area.

optionally, the intelligent terminal is used as an encoder or a decoder, and when it is determined that the current GPM mode is used for performing prediction processing on the block to be processed, after determining the size of the fusion region finally adopted by the block to be processed, the same size of the fusion region may be directly adopted to determine the weight corresponding to each pixel prediction value in the first prediction block and the second prediction block formed by being divided by the partition line in the block to be processed, so that the intelligent terminal may obtain the final prediction value of the current block to be processed by applying the determined weight to the prediction value of the first prediction block and the prediction value of the second prediction block, that is, obtain the prediction block, which is the prediction result of the block to be processed.

S30: and carrying out image coding or decoding processing on the block to be processed according to the prediction result. Alternatively, the image decoding process may be a video encoding process, and the image decoding process may be a video decoding process.

Optionally, after the intelligent terminal is used as an encoder to obtain a prediction result (for example, a prediction block) of a block to be processed currently undergoing a prediction process, the intelligent terminal obtains a residual value of a pixel and a residual block corresponding to the block to be processed by subtracting a prediction value of a corresponding pixel in the prediction block from a sampling value of the pixel in the block to be processed. And then, the intelligent terminal transforms and quantizes the obtained residual block of the block to be processed, and encodes the residual block by the entropy encoder to form the encoded bit stream.

Optionally, the intelligent terminal may also pack, in the encoded bitstream, a prediction mode used for performing prediction processing on the current block to be processed. For example, if the prediction mode used is the GPM mode, the prediction parameters corresponding to the GPM mode are encapsulated and packed in the encoded bitstream together with the side information (side information).

Optionally, the prediction parameter corresponding to the prediction mode includes indication information about the GMP mode.

Optionally, after the intelligent terminal is used as an encoder to obtain a residual block, the residual block and a corresponding prediction block obtained by using the CPM mode may be added to obtain a reconstructed block. Therefore, the intelligent terminal can perform loop filtering processing on the reconstruction block to reduce distortion.

Alternatively, after obtaining a prediction result (e.g., a prediction block) of a block to be processed currently undergoing a prediction process, the intelligent terminal as a decoder may obtain a reconstructed block by adding a corresponding residual block to the prediction block. And then, the reconstruction block is subjected to loop filtering processing through a loop filtering unit to reduce distortion, so that the video quality is improved. And finally, the intelligent terminal can further combine the reconstruction blocks after loop filtering into a decoded image to be stored in a decoded image buffer, or directly output the decoded image as a decoded video signal.

In the embodiment of the application, the intelligent terminal obtains or determines the size of a fusion region of a to-be-processed block which needs to be subjected to prediction processing currently, and then further determines a prediction result of the to-be-processed block, that is, a prediction block according to the size of the fusion region. And finally, carrying out image coding or decoding processing on the block to be processed according to the prediction result. Therefore, the method and the device can directly select the size of the fusion region with high possibility of being used in the process of predicting the image block, and determine the prediction result of the block to be processed based on the size of the fusion region to be used for image coding or decoding processing of the block to be processed, so that the prediction precision of the image is improved, the image coding time is effectively reduced, the overall coding and decoding performance of the video image is improved, and the user experience is further improved.

Second embodiment

Referring to fig. 6, fig. 6 is a flowchart illustrating an image processing method according to a second embodiment of the present application. In this embodiment, the execution subject of the image processing method may be the above-described intelligent terminal.

In the second embodiment of the present application, in the process of performing prediction processing on a block to be processed by using a GPM prediction mode, an intelligent terminal obtains or determines a fusion region size of the block to be processed from a pre-constructed fusion size candidate list by using an image block template in the GPM prediction mode.

Based on this, in the second embodiment of the present application, the above step S10 may include the following steps:

s21: determining a similar template of the image block template in the processed image area;

optionally, when the intelligent terminal obtains or determines the size of the fusion region of the block to be processed by using the image block template in the GPM prediction mode, first, a similar template of the image block template in the GPM prediction mode that is currently used is determined in the encoded or decoded processed image region.

Optionally, the GPM mode adopted by the intelligent terminal to perform prediction processing on the current block to be processed includes a GPM intra-frame prediction mode and a GPM inter-frame prediction mode.

Optionally, when the GPM intra prediction mode is adopted to perform prediction processing on the block to be processed, the intelligent terminal takes an area where all image blocks that have been encoded or decoded in the image where the block to be processed is located as a processed image area, so that the intelligent terminal can determine a similar template of the image block template in the currently adopted GPM intra prediction mode from the processed image area.

Illustratively, the intelligent terminal determines the image blocks above and to the left of the current block to be processed as similar templates of the image block template in the currently adopted GPM intra-frame prediction mode by taking the image area above and the image area on the left of the current block to be processed as the processed image areas in the image in which the current block to be processed is located.

Optionally, when the intelligent terminal performs prediction processing on the block to be processed in the GPM inter-frame prediction mode, the region where all the image blocks that have been encoded or decoded are located in the previous image and/or the subsequent image in the time sequence of the image in which the block to be processed is located is used as the processed image region, and then the similar template of the image block template in the GPM inter-frame prediction mode that is currently used is determined from the processed image region.

For example, the intelligent terminal determines, as the processed image areas, the image blocks located above and left of the image block corresponding to the block to be processed in the previous image and/or the subsequent image of the image in which the block to be processed is located, so that the intelligent terminal further determines, among the processed image areas, the image blocks adjacent to the image block corresponding to the block to be processed above and left as similar templates of the image block template in the currently adopted GPM intra prediction mode.

Optionally, before determining the similar template of the image block template in the currently-used GPM intra prediction mode based on the processed image region, the intelligent terminal further determines a partition line of the current block to be processed, and then determines the similar templates of the image block templates corresponding to different partitions of the block to be processed, which are obtained based on the partition line, from the processed image region based on the partition line, respectively based on the partition line.

For example, the intelligent terminal may specifically determine the division mode with the minimum rate-distortion cost in the 64 division modes by traversing the 64 division modes corresponding to the currently-used GPM prediction mode. And then, the intelligent terminal takes the division mode with the minimum rate distortion cost as a target division mode of the block to be processed, and further determines the type of the partition line corresponding to the target division mode as the partition line of the block to be processed.

Optionally, the 64 partition models corresponding to the GPM prediction mode are shown in table 1 below. Table 1 is a mapping table of the GPM partition index GPM _ partition _ idx, the angle index angleIdx, and the distance index distanceIdx. Based on this, the intelligent terminal finally determines the partitioning mode with the minimum rate distortion cost, wherein the partitioning mode at least comprises the following steps: and determining a target GPM partition index GPM _ partition _ idxT, a target angle index angleIdxT and a target distance index DistanceIdxT corresponding to the partition mode (namely the target partition mode of the block to be processed).

gpm_partition_idx	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
																	angleIdx	0	0	2	2	2	2	3	3	3	3	4	4	4	4	5	5
distanceIdx	1	3	0	1	2	3	0	1	2	3	0	1	2	3	0	1
																	gpm_partition_idx	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31
angleIdx	5	5	8	8	11	11	11	11	12	12	12	12	13	13	13	13
																	distanceIdx	2	3	1	3	0	1	2	3	0	1	2	3	0	1	2	3
gpm_partition_idx	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47
																	angleIdx	14	14	14	14	16	16	18	18	18	19	19	19	20	20	20	21
distanceIdx	0	1	2	3	1	3	1	2	3	1	2	3	1	2	3	1
																	gpm_partition_idx	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62	63
angleIdx	21	21	24	24	27	27	27	28	28	28	29	29	29	30	30	30
																	distanceIdx	2	3	1	3	1	2	3	1	2	3	1	2	3	1	2	3

TABLE 1

Referring to fig. 7, fig. 7 is a schematic diagram of a partition boundary based on the Hessian paradigm according to a second embodiment of the present application. When the angle index angleIdx determined by the intelligent terminal takes different values, corresponding to different angles phi i shown in fig. 7 and fig. 3a, optionally, i is 1 to 24. And when the distance index distanceIdx determined by the intelligent terminal takes different values, ρ j corresponding to fig. 3b and fig. 3c will also be different, and optionally j is 0 to 3.

Optionally, the step S21 may include:

determining a first template and/or a second template of the image block template according to the dividing line, and determining respective similar templates of the first template and/or the second template in the processed image area;

referring to fig. 8, fig. 8 is a schematic diagram of an intra template matching search in a GPM mode according to a second embodiment of the present application. Optionally, when the intelligent terminal determines, from the processed image area, a similar template of an image block template in the currently-used GPM intra-frame prediction mode, the intelligent terminal first uses areas adjacent to the top and left sides of the block to be processed as the image block template in the currently-used GPM intra-frame prediction mode of the block to be processed. Then, the block to be processed (i.e. the current block in the figure) is divided into a partition a and a partition B by a dividing line (as shown by a dotted line in fig. 8), and the extension line of the dividing line also divides the image block template of the block to be processed into a current template a (i.e. the first template) and a current template B (i.e. the second template). Therefore, the intelligent terminal can further respectively determine the similar template A1 and the similar template B1 which are matched with the current template A and the current template B in the processed image area of the image frame where the current block is located.

Optionally, please refer to fig. 9, fig. 9 is a schematic diagram of an interframe template matching search used in the GPM mode according to a second embodiment of the present application. Optionally, when the intelligent terminal determines, from the processed image area, a similar template of the image block template in the currently-used GPM inter-frame prediction mode, the intelligent terminal uses the area above and left adjacent to the image block corresponding to the block to be processed in the previous image and/or the subsequent image in the time sequence of the current image as the image block template of the block to be processed in the currently-used GPM inter-frame prediction mode. Then, similarly, since the to-be-processed block (i.e., the current block in fig. 9) is divided into the partition a and the partition B by the dividing line (as shown by the dotted line in fig. 9), the intelligent terminal divides the image block template of the to-be-processed block into the current template a and the current template B correspondingly in the previous image and/or the next image in the time sequence of the current image, where the extension line of the dividing line is located. Therefore, the intelligent terminal can further respectively determine the similar template A1 and the similar template B1 which are matched with the current template A and the current template B in the processed image area of the previous image and/or the subsequent image of the image where the current block is located.

Optionally, the intelligent terminal may also adopt a GPM intra-frame prediction mode and a GPM inter-frame prediction mode for different image block templates of the block to be processed in the GPM prediction mode at the same time. In this way, if the intelligent terminal applies the template intra-frame prediction to the current template a and applies the template inter-frame prediction to the current template B, the intelligent terminal determines the similar template A1 of the current template a in the processed image area of the image frame where the block to be processed is currently located, and determines the similar template B1 of the current template B in the previous frame and/or the next frame of reference image of the image frame. Similarly, if the intelligent terminal applies the intra-template prediction to the current template B and applies the inter-template prediction to the current template a, the intelligent terminal determines the similar template B1 of the current template B in the processed image region of the image where the block to be processed is currently located, and determines the similar template A1 of the current template a in the previous image and/or the next image (i.e., the reference image) in the temporal sequence of the image.

Optionally, the process of the intelligent terminal determining the similar template of the image block template according to the dividing line is a search process. Illustratively, in the searching process, the intelligent terminal calculates the SSD (SSD refers to the mean square sum of the difference values of the reconstructed block and the source image) between the area and the current template A for the area with the same size as the current template A, and determines the similarity degree between the area and the current template according to the size of the value. If the SSD is less than a threshold, the area is considered to be a similar template A1 to the current template A. Similarly, the terminal device determines the similar template B1 of the current template B by using the same method as the method for determining the similar template A1 of the current template a.

It should be understood that SSD may of course be replaced with SAD or other similar similarity-calculating function in some other possible embodiments, based on different design requirements of the actual application.

S22: applying at least one fusion size to the similar template to obtain or determine the fusion region size.

Optionally, after determining a similar template of the image block template in the GPM prediction mode currently adopted by the intelligent terminal in the encoded or decoded processed image region, the intelligent terminal further applies at least one fusion size in a fusion size candidate list constructed in advance to the similar template, so as to obtain or determine the fusion region size of the to-be-processed block currently required to be subjected to prediction processing in the fusion size candidate list.

Optionally, the step S22 may include:

and determining at least one fusion result according to at least one fusion size in the candidate list of fusion sizes, and acquiring or determining at least one of the at least one fusion size as the size of the fusion region according to the at least one fusion result.

Optionally, when the intelligent terminal applies the fusion size in the candidate fusion size list to the similar template to obtain or determine the size of the fusion area of the block to be processed, first, one or more fusion sizes in the candidate fusion size list are applied to the similar template, so as to determine one or more fusion results corresponding to the fusion size. Then, the intelligent terminal compares the one or more fusion combinations with the image block template of the block to be processed in the currently adopted GPM prediction mode, so as to directly obtain one fusion size in the candidate list of the fusion size according to the comparison result, and the fusion size is used as the fusion area size of the block to be processed, or determine a most possible fusion size from a plurality of fusion sizes in the candidate list of the fusion size according to the comparison result, and the most possible fusion size is used as the fusion area size of the block to be processed.

In the embodiment of the application, when the intelligent terminal obtains or determines the size of the fusion area of the block to be processed by using the image block template in the GPM prediction mode, firstly, a similar template of the image block template in the GPM prediction mode which is currently adopted is determined in the encoded or decoded processed image area. Then, the intelligent terminal further applies at least one fusion size in a pre-constructed fusion size candidate list to the similar template, so as to obtain or determine the fusion area size of the current block to be processed which needs to be subjected to prediction processing in the fusion size candidate list.

Therefore, the image processing method provided by the embodiment of the application can select the size of the fusion region which is likely to be used in advance in the process of predicting the image block by adopting the GPM mode, so that when the GPM mode is adopted, all the fusion sizes of a single image block do not need to be applied and the corresponding rate distortion cost is not calculated, and the time for coding and decoding the video image is effectively reduced.

Third embodiment

Referring to fig. 10, fig. 10 is a flowchart illustrating an image processing method according to a third embodiment of the present application. In this embodiment, the execution subject of the image processing method may be the above-described intelligent terminal.

In a third embodiment of the present application, after determining a similar template of an image block template in the currently employed GPM prediction mode among already encoded or decoded processed image regions according to the process steps set forth in the second embodiment of the present application, the intelligent terminal further determines a similar template comparison region in the similar template, so as to obtain or determine a fusion region size of a block to be processed from a fusion size candidate list based on the similar template comparison region.

Based on this, in the third embodiment of the present application, the above step S10 may include the following steps:

s31: determining a similar template comparison area in the similar template;

optionally, after determining a similar template of the image block template in the currently adopted GPM prediction mode among the already encoded or decoded processed image regions, the intelligent terminal determines a similar template comparison region of the similar template according to the position of the partition line of the block to be processed in the similar template.

Optionally, in addition to determining the similar template comparison area in the similar template, the intelligent terminal also determines the current template comparison area in the image block template of the block to be processed in the GPM prediction mode currently adopted according to the position of the partition line of the block to be processed.

Optionally, the similar templates mentioned above include: a first similar template of the first template of the image block templates and/or a second similar template of the second template of the image block templates. As shown in fig. 8 and/or fig. 9, since the to-be-processed block (i.e., the current block in the drawing) is divided into the partition a and the partition B by the dividing line (as shown by the dotted line in the drawing), the image block template of the to-be-processed block is also divided into the current template a and the current template B by the intelligent terminal, that is, the extension line where the dividing line is located, and the current template a and the current template B are the first template and the second template of the image block template. Optionally, the intelligent terminal further determines a similar template A1 and a similar template B1 that match the current template a and the current template B respectively in the processed image region, that is, a first similar template of the first template and a second similar template of the second template respectively. In another embodiment, there may be a partial overlap region between the current template a and the current template B.

Based on this, the above step S31 may include:

determining a first similar template comparison area according to the image area where the first similar template is located, and/or determining a second similar template comparison area according to the image area where the second similar template is located;

optionally, the intelligent terminal determines a current template comparison area according to the position of the dividing line in an image block template of the block to be processed in the GPM mode, and determines similar template comparison areas in a first similar template of a first template of the image block template and a second similar template of a second template of the image block template, respectively.

Alternatively, the current template comparison area determined in the first template and the second template of the image block template is as shown in fig. 11a, and the similar template comparison area Ac1 of the first similar template of the first template and the similar template comparison area Bc1 of the second similar template of the second template are as shown in fig. 11 b. Optionally, the similar template comparison region Ac1 and the similar template comparison region Bc1 both include extension regions, and if the current template a (i.e., the first template) and the current template B (i.e., the second template) do not have an overlapping region, the extension region of the similar template comparison region Ac1 is located outside the similar template A1, and the extension region of the similar template comparison region Bc1 is located outside the similar template B1.

Optionally, if there is an overlapping region between the current template a and the current template B, the expanded region of the similar template comparison region Ac1 is located in the similar template A1, and the expanded region of the similar template comparison region Bc1 is located in the similar template B1.

Alternatively, the overlapping area of the current template a and the current template B is as shown in fig. 11 c. The current template a is represented by a solid line and the current template B by a dashed line, as is evident: the current template a has an overlapping region with the current template B (i.e., a region where the solid line and the dotted line overlap).

S32: applying at least one fusion size in the fusion size candidate list to the similar template comparison area to obtain at least one fusion result;

optionally, the intelligent terminal determines a plurality of fusion results about the similar template comparison areas by applying different fusion sizes, for example, τ/4, τ/2, τ,2 τ,4 τ,8 τ, respectively, to the similar template comparison areas.

S33: and acquiring or determining the size of the fusion area according to the at least one fusion result.

Optionally, after the intelligent terminal applies the fusion size in the fusion size candidate list to the similar template comparison area to obtain the corresponding fusion result, the intelligent terminal further compares the obtained multiple fusion results with the image block template comparison area of the block to be processed to determine at least one most probable fusion size candidate from different fusion sizes according to each comparison result, and further takes the most probable fusion size candidate as the fusion area size of the block to be processed.

Optionally, the step S33 may include:

and determining a comparison area of the image block template according to the position of the dividing line in the image block template, and comparing the at least one fusion result with the comparison area of the image block template to obtain or determine the size of the fusion area.

Exemplarily, after calculating a plurality of fusion results applying different fusion sizes to the similar template comparison region, the intelligent terminal further compares the plurality of fusion results with the current template comparison region of the image block, that is, calculates a fusion result of each sample in the similar template comparison region Ac1 and the similar template comparison region Bc1, and an SSD (SSD is a sum of mean square of differences between the reconstructed block and the source image) between each sample in the current template comparison region, so as to determine a similarity degree (i.e., a comparison result) between the similar template comparison region and the current template comparison region according to the value. Therefore, the intelligent terminal can select the fusion size corresponding to the minimum similarity degree of the SSD value as the most probable fusion size candidate, and further directly take the most probable fusion size candidate as the fusion area size of the block to be processed.

It should be understood that, based on different design requirements of practical applications, in other possible embodiments, the intelligent terminal may replace the SSD with SAD or other similar similarity-calculating function, so as to perform the comparison between the fusion result and the comparison region of the image block template.

Alternatively, if the intelligent terminal only uses the most probable fusion size candidate determined according to the image block template as the final fusion region size of the block to be processed, and does not use other methods to determine the most probable fusion size candidate, the intelligent terminal does not need to encode the most probable fusion size candidate at the encoding end and pack the most probable fusion size candidate into a bitstream when the intelligent terminal is used as an encoder.

In the embodiment of the application, after the intelligent terminal determines the similar template of the image block template in the currently adopted GPM prediction mode in the encoded or decoded processed image area, the similar template comparison area of the similar template is determined according to the position of the dividing line of the block to be processed in the similar template. Then, the intelligent terminal obtains at least one fusion result by applying at least one fusion size in the fusion size candidate list to the similar template comparison area. And finally, the intelligent terminal compares the obtained multiple fusion results with the image block template comparison area of the block to be processed, so as to determine at least one most probable fusion size candidate from different fusion sizes according to each comparison result, and further take the most probable fusion size candidate as the fusion area size of the block to be processed.

Fourth embodiment

In the fourth embodiment of the present application, the execution subject of the image processing method may be the above-described smart terminal. In this embodiment, the intelligent terminal may determine, according to the image block template, the most likely fusion size candidate as the final adopted fusion region size of the block to be processed, and the intelligent terminal may further obtain or determine at least one target fusion size from a pre-constructed fusion size candidate list, and use the target fusion size as the final adopted fusion region size of the block to be processed; or, the intelligent terminal may further determine at least one most probable fusion size candidate as a fusion area size of the block to be processed according to the fusion size adopted by the adjacent area of the block to be processed; or, the intelligent terminal can also determine at least one most probable fusion size candidate as the fusion area size of the block to be processed according to the fusion size adopted historically; and the intelligent terminal can also determine at least one most probable fusion size candidate as the fusion area size of the block to be processed according to the image block boundary gradient.

Based on this, the step S10 may include at least one of the following ways:

the method I comprises the following steps: acquiring or determining at least one target fusion size from a fusion size candidate list, and taking the target fusion size as the size of the fusion area;

optionally, after the intelligent terminal constructs a candidate list of fusion sizes including at least one fusion size, at least one target fusion size may be directly obtained or determined from the candidate list of fusion sizes, so that the target fusion size is used as a final fusion area size of a block to be processed currently undergoing prediction processing.

Illustratively, the intelligent terminal may obtain the corresponding fusion result by applying a plurality of fusion sizes in the fusion size candidate list to the similar template comparison area. And then, the intelligent terminal can compare the obtained multiple fusion results with the image block template comparison area of the block to be processed, so as to determine at least one target fusion size from different fusion sizes according to each comparison result, and further take the target fusion size as the fusion area size of the block to be processed.

The second method comprises the following steps: applying at least one fusion size to at least one similar template comparison area, calculating by using the respective weight of the at least one similar template comparison area to obtain at least one fusion result, and acquiring or determining the size of the fusion area according to the at least one fusion result;

optionally, after determining a similar template comparison region of a similar template of an image block template in the GPM prediction mode, the intelligent terminal further applies different fusion sizes to the similar template region, so as to determine a plurality of fusion results related to the similar template comparison region by using respective weights of the similar template comparison region, and finally compares the fusion results with the image block template comparison region of the block to be processed based on the plurality of fusion results, so as to determine at least one target fusion size from different fusion sizes according to each comparison result, and further use the target fusion size as the fusion region size of the block to be processed.

Optionally, the weights of the similar template comparison areas are weight W1 and weight W2, respectively. Optionally, the weight W1 is a weight of the first similar template comparison region Ac1, and the weight W2 is a weight of the second similar template comparison region Bc1. The weight W1+ the weight W2 is equal to the maximum value of the weight value range.

Referring to fig. 12a to 12f, fig. 12a, 12b and 12c are distribution diagrams of the weight W1 of the comparison region Ac1 of the similar template. As shown in fig. 12a, 12b, and 12c, fig. 12a corresponds to the case where the fusion size τ is applied to the similar-template comparison region Ac1, fig. 12b corresponds to the case where the fusion size τ is 2 τ, and fig. 12c corresponds to the case where the fusion size τ is 4 τ.

Similarly, fig. 12d, 12e and 12f are distribution diagrams of the values of the weight W2 of the similar template comparison region Bc1. As shown in fig. 12d, 12e and 12f, fig. 12d corresponds to the case where the fusion size is τ, fig. 12e corresponds to the case where the fusion size is 2 τ, and fig. 12f corresponds to the case where the fusion size is 4 τ.

The value ranges of the weight W1 and the weight W2 are 0-16, and the weight W1+ the weight W2 are equal to 16.

It should be understood that the values of the weights W1 and W2 may be other values in other possible embodiments, depending on different design requirements of practical applications. The embodiment of the present application is not limited to the specific values of the weight W1 and the weight W2.

Optionally, the intelligent terminal may set a weight value according to the distance between the similar template comparison region Ac1 and the similar template comparison region Bc1 and the samples (xc, yc) in the similar template comparison region Bc1, and the weight values are set according to the distance between the segmentation lines and different fusion sizes, so as to obtain a weighted prediction sample value predSamples. Illustratively, if the distance from the sample (xca 1, yca 1) to the linear equation in the similar template comparison region Ac1 is larger than the fusion size, the weight W1 corresponding to the sample is set to 0 or 16, and if the distance from the linear equation for the sample (xca 1, yca 1) is smaller than the fusion size, the weight W1 corresponding to the sample is set to k 2. The value range of K2 is 1 to 15. At this time, samples smaller than the fusion size are located near the dividing line.

Alternatively, if the distance to the linear equation for the sample (xcb 1, ycb 1) in the comparison region Bc1 is greater than the fusion size for the similar template, the weight W2 corresponding to the sample is set to 16 or 0, and if the distance to the linear equation for the sample (xcb 1, ycb 1) is less than the fusion size, the weight W2 corresponding to the sample is set to k 2. The value range of K2 is 1 to 15. Also, samples smaller than the blend size are located near the split line.

Optionally, under the condition that different fusion sizes are applied to the similar template comparison region, the intelligent terminal may assign different weight sets to the similar template comparison region Ac1 and the similar template comparison region Bc1. Illustratively, when the fusion size is τ, the set of weights for the fusion region is {4,8, 12}. Alternatively, the samples located on the dividing line correspond to a weight of 8, while the weights located on the two side boundaries (i.e., the blending region boundaries) are 4, 12, respectively. When the fusion size is 2 τ, the set of weights for the fusion region is {2,4,6,8, 10, 12, 14}. Alternatively, the sample located on the dividing line corresponds to a weight of 8, and the weights located on the two side boundaries are 2 and 14, respectively. When the fusion size is 2 τ, the set of weights for the fusion region is {1,2, 3., 13, 14, 15}. Alternatively, the sample located on the dividing line corresponds to a weight of 8, and the weights located on the boundaries on both sides are 1 and 15, respectively.

Thus, the intelligent terminal can further weight the samples in the similar template comparison region Ac1 and the samples in the similar template comparison region Bc1 to obtain the fused predicted value of the samples, that is, according to the following formula (1): s (xc, yc) = f (w 1 × Sac1+ w2 × Sbc 1) fusion result S (xc, yc) obtained by applying the fusion size to the similar template comparison region was calculated.

Alternatively, the above formula (1) may be: s (xc, yc) = W1 × Sac1+ W2 × Sbc1. Herein, sac1 is a fusion result obtained by applying the fusion size to the similar template comparison region Ac1 by the intelligent terminal, and Sbc1 is a fusion result obtained by applying the fusion size to the similar template comparison region Bc1 by the intelligent terminal.

The third method comprises the following steps: acquiring or determining at least one target fusion size according to a preset fusion size of an adjacent image area, and acquiring or determining the size of the fusion area according to the at least one target fusion size;

optionally, when the intelligent terminal determines the size of the fusion region of the to-be-processed block currently predicted in the GPM prediction mode, the intelligent terminal may further determine the adjacent image regions of the to-be-processed block in the processed image region, thereby determining the unique fusion size of each image block in the adjacent image regions, and obtaining or determining at least one target fusion size from each fusion size, and further directly taking the target fusion size as the final adopted fusion region size of the to-be-processed block.

Optionally, the obtaining or determining at least one target fusion size according to the preset fusion size of the adjacent image regions may include:

and determining an adjacent image area corresponding to the block to be processed, taking the fusion size of the adjacent image area as a first reference fusion size, and acquiring or determining at least one target fusion size from a fusion size candidate list according to the first reference fusion size.

Optionally, when the intelligent terminal predicts the current block to be processed by using a GPM intra prediction mode, the intelligent terminal may respectively determine, in a processed image region of the current frame image, left and upper image regions of the block to be processed as adjacent image regions, and then parse, from a bitstream generated by pre-coding, to obtain a fusion size adopted by each image block in the adjacent image region. Therefore, the intelligent terminal can obtain or determine at least one target fusion size from the fusion sizes to serve as the first reference fusion size. Furthermore, the intelligent terminal can directly use the first reference fusion size as the final adopted fusion area size of the block to be processed.

The method is as follows: acquiring or determining the size of the fusion area according to the historical fusion size;

optionally, in the process of performing prediction processing on each image block by using the GPM prediction mode, after performing prediction processing on each image block, the intelligent terminal records the fusion size used by the image block to form a historical fusion size, so that when performing prediction processing on the current block to be processed by using the GPM prediction mode, the intelligent terminal may directly obtain or determine a most likely fusion size candidate from one or more historical fusion sizes to serve as the final fusion area size used by the block to be processed.

Optionally, the intelligent terminal obtains or determines a fusion area size of the block to be processed according to the historical fusion size, where the fusion area size at least includes at least one of the following:

acquiring or determining a second reference fusion size meeting the use time condition from a historical fusion size queue, and acquiring or determining the size of the fusion area according to the second reference fusion size;

and acquiring or determining a third reference fusion size meeting the enqueue time condition from the historical fusion size queue, and acquiring or determining the fusion area size according to the third reference fusion size.

Optionally, the intelligent terminal constructs a historical fusion size queue in advance in the process of performing prediction processing on each image block in the GPM prediction mode, so that after the prediction processing is performed on each image block, the fusion size adopted by the image block is recorded in the historical fusion size queue. Moreover, when the intelligent terminal performs prediction processing on two image blocks, if the two image blocks both adopt the same fusion size, the number of times of use of the fusion size adopted by any one image can be overlapped in the historical fusion size queue by the intelligent terminal is two. Furthermore, when the intelligent terminal performs prediction processing on the current block to be processed by adopting the GPM prediction mode, one fusion size with the largest recorded use times can be selected from the historical fusion size queue as a second reference fusion size meeting the use time condition, and the second reference fusion size is further determined as the currently adopted fusion area size of the block to be processed.

Optionally, the intelligent terminal constructs a historical fusion size queue, so that after each image block is subjected to prediction processing, when the fusion size adopted by the image block is recorded in the historical fusion size queue, the recording time for recording the fusion size adopted by each image block into the historical fusion size queue is also synchronously recorded. Furthermore, when the intelligent terminal performs prediction processing on the current block to be processed by adopting the GPM prediction mode, a more fusion size recorded with the last enqueue time is selected from the historical fusion size queue as a third reference fusion size meeting the enqueue time condition, and the third reference fusion size is further determined as the currently adopted fusion area size of the block to be processed.

Optionally, when the intelligent terminal constructs a historical fusion size queue and performs prediction processing on each image block by using a GPM prediction mode, the fusion size adopted by each image block is recorded, and the fusion size may include at least one of the following:

constructing a historical fusion size queue according to the fusion area size corresponding to at least one processed block in the processed image area;

and recording the using times and the enqueue time of at least one fused size in the historical fused size queue.

Optionally, the intelligent terminal constructs a historical fusion size queue in advance in the process of performing prediction processing on each image block by using the GPM prediction mode, so that after prediction processing is performed on each image block, the fusion size adopted by the processed image block (i.e., the processed block) is recorded in the historical fusion size queue. Moreover, when the intelligent terminal performs prediction processing on two image blocks, if the two image blocks both adopt the same fusion size, the number of times of use of the fusion size adopted by any one image can be overlapped in the historical fusion size queue by the intelligent terminal is two. And when the intelligent terminal records the fusion size adopted by the image block subjected to prediction processing in the historical fusion size queue, the intelligent terminal also synchronously records the recording time (namely the queuing time) of the fusion size adopted by each image block recorded in the historical fusion size queue.

The fifth mode is as follows: and acquiring or determining the size of the fusion region according to a preset image block boundary gradient.

Optionally, in the process of performing prediction processing on the block to be processed by using the GPM prediction mode, the intelligent terminal may further obtain or determine a size of a fusion region finally adopted by the block to be processed according to an image block boundary gradient corresponding to the block to be processed.

Optionally, before determining the size of the fusion region adopted by the block to be processed according to the image block boundary gradient, the intelligent terminal further determines the image block boundary gradient according to the partition line.

Optionally, the intelligent terminal determines the image block boundary gradient according to the dividing line, and may include:

and determining image samples passed by the straight line where the dividing line is located in the processed image area, and taking the gradient between the image samples and the adjacent samples of the image samples as the boundary gradient of the image block.

Referring to fig. 13, fig. 13 is a schematic diagram of a sample passed by a straight line where a GPM dividing line is located according to a fourth embodiment of the present invention. The intelligent terminal determines the straight line where the GPM dividing line is located in the processed area by adopting the GPM prediction mode (as shown by the oblique shaded part in FIG. 13); then, the intelligent terminal can determine the gradient between the sample within the preset distance from the dividing line and the sample adjacent to the sample as the image block boundary gradient. Therefore, the intelligent terminal obtains or determines the size of the fusion area adopted by the block to be processed according to the boundary gradient of the image block.

Illustratively, the intelligent terminal as an encoder can use the encoded region as an adjacent image region formed by sampling adjacent rows or left adjacent columns on the block to be processed. Optionally, if the distance from one sample in the encoded region to the straight line where the GPM partition line is located is less than 1/2 of the sample, the intelligent terminal determines the sample as the sample through which the straight line where the GPM partition line is located passes. Or, in another possible embodiment, if a sample in the encoded region is less than 1 ≦ based on the distance from the line on which the GPM partition line resides

And sampling, and the intelligent terminal also determines the sampling as the sampling passed by the straight line where the GPM dividing line is located.

After determining the samples passed by the straight line of the GPM dividing line, the intelligent terminal further calculates the horizontal gradient of a predetermined number of samples adjacent to the horizontal direction of the samples and/or calculates the vertical gradient of a predetermined number of samples adjacent to the vertical direction of the samples. Therefore, the intelligent terminal can obtain or determine at least one most probable fusion size candidate according to the coordinate range by determining the sampling coordinate range of which the square sum of the horizontal gradient and the vertical gradient is greater than the preset threshold value, and then the intelligent terminal determines the most probable fusion size candidate as the final adopted fusion area size of the block to be processed.

In the embodiment of the application, when the intelligent terminal determines the size of the fusion area adopted by the block to be processed according to the boundary gradient of the image block, the coordinate range of the sample with the square sum of the horizontal gradient and the vertical gradient larger than the predetermined threshold value selected by the intelligent terminal is equivalent to the range of the sample with obvious change near the GPM dividing line selected by the intelligent terminal. Since the features of the coded area adjacent to the image block are similar to those of the image block, the fusion size corresponding to the sampling range is more likely to be suitable for the current GPM segmentation mode than other fusion sizes.

Referring to fig. 14, fig. 14 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application, where the image processing apparatus 140 includes: a first determining module 1401, a second determining module 1402 and a processing module 1403.

A first determining module 1401, configured to obtain or determine a size of a fusion region of a block to be processed;

a second determining module 1402, configured to determine a prediction result of the block to be processed according to the size of the fusion region;

a processing module 1403, configured to perform image encoding or decoding processing on the block to be processed according to the prediction result.

Optionally, the first determining module 1401 is further configured to obtain or determine the size of the fusion region according to a preset image block template.

Optionally, the preset image block template is an image block template when the block to be processed uses a template-based prediction mode; and/or, the first determining module 1401, comprising:

a template determination unit for determining a similar template of the image block template in the processed image area;

a first size determination unit for applying at least one fusion size to the similar template to obtain or determine the fusion region size.

Optionally, the template determining unit is further configured to determine a first template and/or a second template of the image block template according to the dividing line, and determine a similar template of each of the first template and/or the second template in the processed image area;

the first size determining unit is further configured to determine at least one fusion result according to at least one fusion size in the candidate fusion size list, and obtain or determine at least one of the at least one fusion size as the fusion region size according to the at least one fusion result.

Optionally, the first determining module 1401 further includes:

a comparison area determination unit for determining a similar template comparison area in the similar template;

the region fusion unit is used for applying at least one fusion size in the fusion size candidate list to the similar template comparison region to obtain at least one fusion result;

and the second size determining unit is used for acquiring or determining the size of the fusion area according to the at least one fusion result.

Optionally, the similar templates include: the comparison area determining unit is further used for determining a comparison area of the first similar template according to the image area where the first similar template is located, and/or determining a comparison area of the second similar template according to the image area where the second similar template is located;

the second size determining unit is further configured to determine an image block template comparison area according to the position of the dividing line in the image block template, and compare the at least one fusion result with the image block template comparison area to obtain or determine the size of the fusion area.

Optionally, the first determining module 1401 further includes:

a third size determining unit, configured to obtain or determine at least one target fusion size from the candidate fusion size list, and use the target fusion size as the fusion region size;

the fourth size determining unit is used for applying at least one fusion size to at least one similar template comparison area, calculating by using the respective weight of the at least one similar template comparison area to obtain at least one fusion result, and acquiring or determining the size of the fusion area according to the at least one fusion result;

a fifth size determining unit, configured to obtain or determine at least one target fusion size according to a preset fusion size of an adjacent image region, and obtain or determine the size of the fusion region according to the at least one target fusion size;

a sixth size determining unit, configured to obtain or determine the size of the fusion region according to a historical fusion size;

and the seventh size determining unit is used for acquiring or determining the size of the fusion area according to a preset image block boundary gradient.

Optionally, the fifth size determining unit is further configured to determine an adjacent image region corresponding to the block to be processed, use a fusion size of the adjacent image region as a first reference fusion size, and obtain or determine at least one target fusion size from a fusion size candidate list according to the first reference fusion size.

Optionally, the sixth size determining unit is further configured to obtain or determine a second reference fusion size meeting the condition of the number of times of use from the historical fusion size queue, and obtain or determine the size of the fusion region according to the second reference fusion size; and/or acquiring or determining a third reference fusion size meeting the enqueue time condition from the historical fusion size queue, and acquiring or determining the fusion area size according to the third reference fusion size.

Optionally, the image processing apparatus further includes a construction module and a recording module.

The construction module is used for constructing a historical fusion size queue according to the fusion area size corresponding to at least one processed block in the processed image area;

the recording module is used for recording the using times and the enqueue time of at least one fusion size in the historical fusion size queue.

Optionally, the image processing apparatus of the present application further includes a gradient determination module.

The gradient determining module is used for determining the boundary gradient of the image block according to the dividing line.

Optionally, the gradient determining module is further configured to determine an image sample that a straight line where the dividing line is located passes through in the processed image region, and use a gradient between the image sample and an adjacent sample of the image sample as an image block boundary gradient.

It should be understood that the functions of the functional modules of the image processing apparatus described in the embodiment of the present application may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the relevant description of the foregoing method embodiment, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail.

The embodiment of the application also provides an image processing method which can be applied to an intelligent terminal, and the image processing method comprises the following steps:

Optionally, the step S10 includes:

Optionally, the preset image block template is an image block template when the to-be-processed block uses a template-based prediction mode.

Optionally, the step S10 includes the steps of:

Optionally, the method comprises at least one of:

the step S21 includes: determining a first template and/or a second template of the image block template according to the dividing line, and determining similar templates of the first template and/or the second template in the processed image area;

the step of S22 includes: and determining at least one fusion result according to at least one fusion size in the candidate list of fusion sizes, and acquiring or determining at least one of the at least one fusion size as the size of the fusion region according to the at least one fusion result.

Optionally, the step S10 includes the steps of:

s31: determining a similar template comparison area in the similar template;

s32: applying at least one fusion size in the candidate fusion size list to the similar template comparison area to obtain at least one fusion result;

Optionally, the method comprises at least one of:

the similar template comprises: the first similar template of the first template of the image block template and/or the second similar template of the second template of the image block template, and/or the step S31, includes: determining a first similar template comparison area according to the image area where the first similar template is located, and/or determining a second similar template comparison area according to the image area where the second similar template is located;

the step S33 includes: and determining a comparison area of the image block template according to the position of the dividing line in the image block template, and comparing the at least one fusion result with the comparison area of the image block template to obtain or determine the size of the fusion area.

Optionally, the S10 step includes at least one of:

acquiring or determining at least one target fusion size from a fusion size candidate list, and taking the target fusion size as the size of the fusion area;

applying at least one fusion size to at least one similar template comparison area, calculating by using the respective weight of the at least one similar template comparison area to obtain at least one fusion result, and acquiring or determining the size of the fusion area according to the at least one fusion result;

acquiring or determining at least one target fusion size according to a preset fusion size of an adjacent image area, and acquiring or determining the size of the fusion area according to the at least one target fusion size;

acquiring or determining the size of the fusion area according to the historical fusion size;

and acquiring or determining the size of the fusion region according to a preset image block boundary gradient.

Optionally, the obtaining or determining at least one target fusion size according to a preset fusion size of adjacent image regions includes:

Optionally, the method comprises at least one of:

Optionally, the method further comprises at least one of:

Optionally, before the obtaining or determining the size of the fusion region according to the preset image block boundary gradient, the method further includes:

and determining the boundary gradient of the image block according to the dividing line.

Optionally, the determining the image block boundary gradient according to the partition line includes:

It should be understood that, for the image processing method described in the embodiments of the present application, the specific implementation process thereof may refer to the description related to the first to fourth embodiments, and details are not described herein again. In addition, the beneficial effects of the same method are described, and are not repeated herein.

The embodiment of the present application further provides an intelligent terminal, where the intelligent terminal includes a memory and a processor, and the memory stores an image processing program, and the image processing program is executed by the processor to implement the steps of the image processing method in any of the above embodiments. The intelligent terminal provided by the embodiment of the application can be a mobile terminal 100 as shown in fig. 1.

Alternatively, the processor 110 of the mobile terminal 100 as shown in fig. 1 may be used to call up an image processing program stored in the memory 109 to perform the following operations: s10: acquiring or determining the size of a fusion area of a block to be processed; s20: determining a prediction result of the block to be processed according to the size of the fusion area; s30: and carrying out image coding or decoding processing on the block to be processed according to the prediction result.

Optionally, the processor 110 of the mobile terminal 100 may be further configured to call an image processing program stored in the memory 109 to perform the following operations:

Optionally, the preset image block template is an image block template when the block to be processed uses a template-based prediction mode; the processor 110 of the mobile terminal 100 may also be configured to invoke an image processing program stored in the memory 109 to perform the following operations:

determining a first template and/or a second template of the image block template according to the dividing line, and determining similar templates of the first template and/or the second template in the processed image area;

s31: determining a similar template comparison area in the similar template;

Optionally, the similar templates include: presetting a first similar template of a first template of image block templates and/or a second similar template of a second template of said image block templates, the processor 110 of the mobile terminal 100 may be further configured to invoke an image processing program stored in the memory 109 to perform the following operations:

Optionally, the processor 110 of the mobile terminal 100 may be further configured to call an image processing program stored in the memory 109, and before performing the obtaining or determining the size of the fusion region according to the preset image block boundary gradient, further perform the following operations:

determining image samples passing by the straight line where the dividing line is located in the processed image area, and taking the gradient between the image samples and the adjacent samples of the image samples as the boundary gradient of the image block.

It should be understood that the mobile terminal described in the embodiment of the present application may perform the description of the image processing method in any one of the embodiments described above, and may also perform the description of the image processing apparatus in the corresponding embodiment described above, which is not described herein again. In addition, the beneficial effects of the same method are not described in detail herein.

The embodiment of the present application further provides a storage medium, where an image processing program is stored on the storage medium, and the image processing program, when executed by a processor, implements the steps of the image processing method in any of the above embodiments.

In the embodiments of the intelligent terminal and the storage medium provided in the present application, all technical features of any one of the embodiments of the image processing method may be included, and the expanding and explaining contents of the specification are basically the same as those of the embodiments of the method, and are not described herein again.

Embodiments of the present application also provide a computer program product, which includes computer program code, when the computer program code runs on a computer, the computer is caused to execute the method in the above various possible embodiments.

Embodiments of the present application further provide a chip, which includes a memory and a processor, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device in which the chip is installed executes the method in the above various possible embodiments.

It is to be understood that the foregoing scenarios are only examples, and do not constitute a limitation on application scenarios of the technical solutions provided in the embodiments of the present application, and the technical solutions of the present application may also be applied to other scenarios. For example, as a person having ordinary skill in the art can know, with the evolution of the system architecture and the emergence of new service scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.

The units in the device in the embodiment of the application can be merged, divided and deleted according to actual needs.

In the present application, the same or similar term concepts, technical solutions and/or application scenario descriptions will be generally described only in detail at the first occurrence, and when the description is repeated later, the detailed description will not be repeated in general for brevity, and when understanding the technical solutions and the like of the present application, reference may be made to the related detailed description before the description for the same or similar term concepts, technical solutions and/or application scenario descriptions and the like which are not described in detail later.

In the present application, each embodiment is described with an emphasis on the description, and reference may be made to the description of other embodiments for parts that are not described or recited in any embodiment.

The technical features of the technical solution of the present application may be arbitrarily combined, and for brevity of description, all possible combinations of the technical features in the embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, the scope of the present application should be considered as being described in the present application.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application or portions contributing to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (such as a ROM/RAM, a magnetic disk, and an optical disk) as above, and includes several instructions to enable a terminal device (which may be a mobile phone, a computer, a server, a controlled terminal, or a network device) to execute the method of each embodiment of the present application.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a storage medium or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., floppy Disk, memory Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims

1. An image processing method, characterized by comprising the steps of:

2. The method of claim 1, wherein the S10 step includes:

3. The method of claim 2, wherein the predetermined image block template is an image block template when the block to be processed uses a template-based prediction mode; and/or, the step S10 includes the steps of:

4. The method of claim 3, wherein the method comprises at least one of:

the step S22 includes: and determining at least one fusion result according to at least one fusion size in the fusion size candidate list, and acquiring or determining at least one of the at least one fusion size as the fusion area size according to the at least one fusion result.

5. The method of claim 1, wherein the S10 step includes the steps of:

s31: determining a similar template comparison area in the similar template;

6. The method of claim 5, wherein the method comprises at least one of:

the similar template comprises: presetting a first similar template of a first template of an image block template and/or a second similar template of a second template of the image block template, and/or the step S31 includes: determining a first similar template comparison area according to the image area where the first similar template is located, and/or determining a second similar template comparison area according to the image area where the second similar template is located;

the step S33 includes: and determining an image block template comparison area according to the position of the division line in the image block template, and comparing the at least one fusion result with the image block template comparison area to obtain or determine the size of the fusion area.

7. The method of claim 1, wherein the S10 step includes at least one of:

determining the fusion region size from a plurality of fusion sizes;

8. The method as claimed in claim 7, wherein the obtaining or determining at least one target fusion size according to the preset fusion size of the adjacent image area comprises:

9. The method of claim 7, wherein the method comprises at least one of:

10. The method of claim 7 or 9, wherein the method further comprises at least one of:

11. The method as claimed in claim 7, wherein before said obtaining or determining the size of the fusion region according to the preset image patch boundary gradient, the method further comprises:

12. The method of claim 11, wherein determining tile boundary gradients from partition lines comprises:

13. An intelligent terminal, comprising: memory, a processor, wherein the memory has stored thereon an image processing program which, when executed by the processor, implements the steps of the image processing method of any one of claims 1 to 12.

14. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the image processing method according to any one of claims 1 to 12.