WO2018072716A1 - 图像处理方法及装置 - Google Patents

图像处理方法及装置 Download PDF

Info

Publication number
WO2018072716A1
WO2018072716A1 PCT/CN2017/106740 CN2017106740W WO2018072716A1 WO 2018072716 A1 WO2018072716 A1 WO 2018072716A1 CN 2017106740 W CN2017106740 W CN 2017106740W WO 2018072716 A1 WO2018072716 A1 WO 2018072716A1
Authority
WO
WIPO (PCT)
Prior art keywords
video image
video
adjusting
parameter
fluency
Prior art date
Application number
PCT/CN2017/106740
Other languages
English (en)
French (fr)
Inventor
余立艳
陈虹宇
Original Assignee
西安中兴新软件有限责任公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西安中兴新软件有限责任公司 filed Critical 西安中兴新软件有限责任公司
Publication of WO2018072716A1 publication Critical patent/WO2018072716A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working

Definitions

  • This document relates to the field of communications, and in particular to an image processing method and apparatus.
  • Embodiments of the present invention provide an image processing method and apparatus, which are capable of at least adjusting at least one of clarity and fluency of a video image during a video call.
  • an image processing method including: transmitting a video image to a called device; receiving a notification message that the called device feeds back according to the video image, wherein the notification message carries the An image adjustment parameter of the video image, the image adjustment parameter including at least one of a sharpness parameter and a fluency parameter; and adjusting a video image during a video call according to the image adjustment parameter.
  • adjusting the video image during the video call according to the image adjustment parameter includes: when the notification message carries the sharpness parameter of the video image, determining, according to the definition parameter, the clarity of the video image during the video call Whether the degree reaches the first predetermined value; when the notification message carries the fluency parameter of the video image, determining, according to the fluency parameter, whether the fluency of the video image during the video call reaches a second predetermined value; No, the corresponding adjustment At least one of clarity and fluency of the video image during the video call.
  • the adjusting the resolution of the video image during the video call comprises: adjusting a reservation for controlling the definition of the video image. Adjusting parameters; adjusting the sharpness of the video image during the video call according to the adjusted predetermined adjustment parameter.
  • the adjusting the fluency of the video image during the video call includes: adjusting a macroblock division manner of the video image, Adjusting the fluency of the video image during the video call, wherein one macroblock includes one luma pixel block and two additional chroma pixel blocks.
  • the method further includes: determining whether the fluency of the adjusted video image reaches a second predetermined value; In the case that the determination result is negative, the fluency of the video image during the video call is adjusted by adjusting the range of the motion vector MV, wherein the MV is the current coding of the video image during inter prediction.
  • MV is the current coding of the video image during inter prediction.
  • an image processing apparatus comprising: a transmitting module configured to transmit a video image to a called device; and a receiving module configured to receive a notification message fed back by the called device according to the video image
  • the notification message carries an image adjustment parameter of the video image, the image adjustment parameter includes at least one of a sharpness parameter and a fluency parameter of the video image; and an adjustment module configured to be based on the image Adjust parameters to adjust the video image during a video call.
  • the adjusting module is configured to determine, according to the definition parameter, whether the definition of the video image during the video call reaches a first predetermined value when the image adjustment parameter includes a sharpness parameter; When the parameter includes the fluency parameter, determining whether the fluency of the video image during the video call reaches a second predetermined value according to the fluency parameter; and if the determination result is no, adjusting the video call process correspondingly At least one of the clarity and fluency of the video image.
  • the adjusting module is configured to adjust the clarity of the video image during the video call by determining when adjusting the resolution of the video image during the video call Degree: adjusting a predetermined adjustment parameter for controlling the sharpness of the video image; and adjusting a sharpness of the video image during the video call according to the adjusted predetermined adjustment parameter.
  • the adjusting module is configured to adjust the fluency of the video image during the video call by adjusting the video when determining that the video image is smooth during the video call:
  • the manner of macroblock division of the image adjusts the fluency of the video image during the video call, wherein one macroblock includes one luma pixel block and two additional chroma pixel blocks.
  • the adjusting module is further configured to determine whether the fluency of the adjusted video image is reached after adjusting a fluency of the video image during the video call by adjusting a macroblock division manner of the video image. a second predetermined value; and in the case of a negative determination result, the fluency of the video image during the video call is adjusted by adjusting a range of the motion vector MV, wherein the MV is an interframe prediction a motion vector between a currently coded macroblock of the video image and a macroblock of the reference frame that has the highest degree of matching with the currently coded macroblock.
  • a storage medium is also provided.
  • the storage medium is configured to store program code for performing the steps of: transmitting a video image to the called device; receiving a notification message that the called device feeds back according to the video image, wherein the notification message carries the video image And an image adjustment parameter, where the image adjustment parameter includes at least one of a sharpness parameter and a fluency parameter; and adjusting a video image during a video call according to the image adjustment parameter.
  • the storage medium is further configured to store program code for performing the following steps: adjusting the video image during the video call according to the image adjustment parameter comprises: when the notification message carries the sharpness parameter of the video image, according to the Determining whether the resolution of the video image reaches the first predetermined value during the video call; and when the notification message carries the fluency parameter of the video image, determining the video during the video call according to the fluency parameter Whether the fluency of the image reaches a second predetermined value; if the determination result is no, at least one of the sharpness and the fluency of the video image during the video call is correspondingly adjusted.
  • the storage medium is further configured to store program code for performing the following steps: adjusting the definition of the video image during the video call when it is determined to adjust the definition of the video image during the video call
  • the method includes: adjusting a predetermined adjustment parameter for controlling the definition of the video image; and adjusting a video image during the video call according to the adjusted predetermined adjustment parameter The clarity.
  • the storage medium is further configured to store program code for performing the following steps: adjusting the fluency of the video image during the video call when determining to adjust the fluency of the video image during the video call
  • the method includes: adjusting a fluency of a video image during the video call by adjusting a macroblock division of the video image, where one macroblock includes one luma pixel block and two additional chroma pixel blocks.
  • the storage medium is further configured to store program code for performing the following steps: after adjusting the fluency of the video image during the video call by adjusting a macroblock division manner of the video image, the method further includes: determining Whether the smoothness of the adjusted video image reaches a second predetermined value; if the determination result is negative, adjusting the fluency of the video image during the video call by adjusting the range of the motion vector MV, wherein The MV is a motion vector between a currently coded macroblock of the video image and a macroblock of the reference frame that has the highest matching degree with the current coded macroblock.
  • a video image is sent to the called device; and the notification message fed back by the called device according to the video image is received, wherein the notification message carries an image adjustment parameter of the video image, and the image adjustment parameter includes a sharpness parameter and a smoothness At least one of the degree parameters; adjusting the video image during the video call according to the image adjustment parameter. At least one of the sharpness parameter and the fluency of the video image during the video call is adjusted according to at least one of the sharpness parameter of the video image transmitted by the called device and the smoothness parameter of the video image. Therefore, the embodiment of the present invention can adjust at least one of the definition and the smoothness of the video image during the video call.
  • FIG. 1 is a block diagram showing the hardware structure of a mobile terminal of an image processing method according to an embodiment of the present invention
  • FIG. 2 is a flow chart of an image processing method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a video call mobile terminal based on H264 encoding according to an embodiment of the present invention
  • FIG. 4 is a schematic diagram of partitioning of H264 macroblocks and subblocks according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of a video call of an H264 encoder according to an embodiment of the present invention.
  • FIG. 6 is a block diagram showing the structure of an image processing apparatus according to an embodiment of the present invention.
  • H264 algorithms Most of the algorithms used in the video call process are H264 algorithms.
  • the H264 protocol has the concept of layered design, and video encoders in different scenarios can adopt different technical algorithms.
  • the baseline profile algorithm is applied to a simple scenario, and the main profile or the high profile algorithm is applied to a complex scenario.
  • the H264 encoder used on the terminal has always been the encoding algorithm of the baseline profile specification, but with the development of the terminal processor and related hardware technologies, the video encoder of the terminal will not be limited to
  • the algorithm of the baseline profile may use the encoding algorithm of the main profile specification.
  • FIG. 1 is a block diagram showing a hardware structure of a mobile terminal according to an image processing method according to an embodiment of the present invention.
  • the mobile terminal 10 may include one or more (only one shown in the figure) processor 102 (the processor 102 may include a processing device such as a microprocessor MCU or a programmable logic device FPGA), settings A memory 104 for storing data and a transmission device 106 provided as a communication function.
  • the structure shown in FIG. 1 is merely illustrative and does not limit the structure of the above electronic device.
  • the mobile terminal 10 may also include more or fewer components than those shown in FIG. 1, or have a different configuration than that shown in FIG.
  • the memory 104 can be configured as a software program and a module for storing application software, as embodied by the present invention
  • the program instruction or module corresponding to the image processing method in the example, the processor 102 is configured to execute a plurality of function applications and data processing by running a software program and a module stored in the memory 104, that is, to implement the above method.
  • Memory 104 may include high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory.
  • memory 104 may further include memory remotely located relative to processor 102, which may be connected to mobile terminal 10 over a network. Examples of such networks include the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Transmission device 106 is arranged to receive or transmit data via a network.
  • the network instance described above may include a wireless network provided by a communication provider of the mobile terminal 10.
  • the transmission device 106 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet.
  • the transmission device 106 can be a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.
  • NIC Network Interface Controller
  • RF Radio Frequency
  • FIG. 2 is a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 2, the flow includes the following steps:
  • Step S202 sending a video image to the called device
  • Step S204 receiving a notification message fed back by the called device according to the video image, where the notification message carries an image adjustment parameter of the video image, and the image adjustment parameter includes at least one of a sharpness parameter and a fluency parameter of the video image;
  • Step S206 adjusting a video image during a video call according to the image adjustment parameter, such as adjusting a sharpness of a video image during a video call according to a sharpness parameter, and adjusting a smoothness of a video image during a video call according to a fluency parameter.
  • the image adjustment parameter such as adjusting a sharpness of a video image during a video call according to a sharpness parameter, and adjusting a smoothness of a video image during a video call according to a fluency parameter.
  • At least one of the sharpness parameter of the video image sent by the called device and the fluency parameter of the video image is adjusted to adjust at least one of the sharpness and the smoothness of the video image during the video call.
  • adjusting the video image during the video call according to the image adjustment parameter includes: when the notification message carries the sharpness parameter of the video image, determining the video call according to the sharpness parameter Whether the resolution of the video image reaches the first predetermined value during the process; when the notification message carries the fluency parameter of the video image, it is determined according to the fluency parameter whether the fluency of the video image during the video call reaches a second predetermined value; If the result is no, at least one of the definition and the smoothness of the video image during the video call is adjusted correspondingly.
  • adjusting the resolution of the video image during the video call comprises: adjusting a predetermined adjustment parameter for controlling the definition of the video image; and adjusting the sharpness of the video image during the video call according to the adjusted predetermined adjustment parameter.
  • adjusting the fluency of the video image during the video call includes: adjusting the fluency of the video image during the video call by adjusting a macroblock division of the video image, where one macroblock includes a luma pixel block and an additional Two chrominance pixel blocks.
  • the method further includes: determining whether the fluency of the adjusted video image reaches a second predetermined value; In the case of no, the fluency of the video image during the video call is adjusted by adjusting the range of the motion vector (Motion Vector, MV for short), wherein the MV is the current coded macroblock of the video image during inter prediction.
  • Motion Vector Motion Vector
  • the embodiment of the present invention provides a video call mobile terminal based on the H264 code.
  • the user can ensure clear and smooth picture quality during the video call on the mobile terminal. If the picture is not clear, the user can manually adjust the video quality and picture fluency.
  • 3 is a schematic diagram of a video call mobile terminal based on H264 encoding according to an embodiment of the present invention.
  • the H264-encoded video call mobile terminal includes: a video telephony module, a parameter configuration module, a video adjustment module, and a network standard. Monitoring module, H264 encoder module.
  • the video telephony module is configured to, when initiating or answering the video call, deliver an indication of a video call to the bottom layer, the indication flag indicating that the current scene is a video call scenario.
  • the parameter configuration module is set to save the parameter values that the encoder needs to set at the beginning and during the encoding.
  • the parameter values can be divided into the parameter values of the three specification algorithms, which are the parameter values of the Baseline Profile, Main Profile, and High Profile algorithms.
  • Each specification algorithm consists of Base Params and Dynamic Params parameters.
  • Base Params is set when the encoder is created. Parameter, this part of the parameter value is generally set to the minimum reference value.
  • Dynamic Params is a parameter that is dynamically set according to the change of the video source during the operation of the encoder. This parameter is set before each frame starts encoding. When the indicator transmitted by the upper layer indicates that the current user is initiating or receiving a video call, the video encoder is started and the parameter value of the Base line is selected.
  • the video adjustment module is configured to adjust the parameters of the encoder according to the quality and fluency of the video picture fed back by the called party, and adjust the video quality and fluency during the video call in real time. Adjusting parameters includes adjusting the encodingPreset parameter, adjusting the division mode of the macroblock, and adjusting the range of the MV (Motion vector).
  • the encodingPreset parameter is used to control the picture quality during a video call. It can be divided into high, medium, and low. When the selection is high, the video encoding resolution will be high, the corresponding stream data will be large, and the compression ratio will be low.
  • H264 Adjust video fluency by adjusting the macroblock division.
  • the H264 protocol has improved compression rate and improved algorithm complexity. The most important improvement is that the minimum macroblock partitioning can be refined to 4X4, while the H263 macroblock partitioning is 8X8.
  • H264 has 7 macroblock partition modes, which can be divided according to 16X16, 16X8, 8X16, 8X8: If you choose 8X8 macroblock mode, you can follow 8X4. , 4X8, 4X4 for sub-macroblock segmentation. The division of macroblocks and subblocks is shown in Figure 4.
  • H.264 macroblock and sub-block partitioning What type of macroblock partitioning is used in encoding, each macroblock can be divided into different types by trial algorithm, and then the absolute error and value are calculated (Sum Absolute Difference, SAD for short) ), finally select the best way to divide the macroblock. Due to the variety of H264 macroblock partitioning, each algorithm traverses once and then finds the optimal partitioning method, which is time consuming and does not take advantage of the fluency of the video. During the video call, the picture is generally simple, the main body is the character, and the change of the style is generally not too big. The most change is the eyes and mouth.
  • the % macroblock partition is 1X1, and only 9% is likely to be divided into 16X16. Therefore, in the video encoding process, the macroblock division type can be adjusted to increase or slow down the encoding speed, and the encoding speed is reflected in the smoothness of the video.
  • MV Motion Vector
  • the fluency, MV is the motion vector between the currently coded macroblock and the highest matching macroblock in the reference frame during inter prediction.
  • the most well-matched macroblock search method can calculate a macroblock and a macroblock in a reference frame, and then find the best matching macroblock, so that one adaptation calculation affects the encoding speed. If a search range is determined during macroblock prediction during the encoding process, and the matching macroblocks are searched within a certain range, the adaptation time can be reduced, which is advantageous for improving the encoding speed.
  • the range of the MV can be set to several gear positions.
  • the video call process changes simply, so each macro block does not need to match the entire reference frame, and a matching macroblock may be found in a small range, so in coding
  • the range of the MV can be set to a slightly lower gear position, the search range of the matching macroblock is reduced, the encoding time is reduced, the encoding speed is improved, and the video smoothness is improved.
  • the network standard monitoring module is configured to monitor the network standard registered by the current user, and then feed back to the video adjustment module. Due to different network standards and different video bandwidth limitations, the parameters of the video call (encodingPreset, macroblock division, and MV range) are adjusted.
  • the H264 encoder is configured to encode the forward-introduced image data according to the parameter configuration module and the user-adjusted configuration parameters.
  • FIG. 5 is a flowchart of a video call of an H264 encoder according to an embodiment of the present invention. As shown in FIG. 5, the process includes:
  • Step S502 The user initiates or answers a video call, and delivers a video call indication flag to the bottom layer, indicating that the current scene is a video call scenario.
  • Step S504 the encoder configuration module receives the upper layer delivery video indication flag, and selects a Baseline specification algorithm in the configuration module.
  • Step S506 After the Baseline specification algorithm is determined, the initial value of the encoder is configured, and the initial value of the encoder is Base Params, and the part of the parameter may be set to a minimum value.
  • Dynamic Params is a parameter that is dynamically set according to the change of the video source during the operation of the encoder. This parameter needs to be set before each frame starts encoding.
  • step S508 when the called video image of the calling party is not clear, the process goes to step S510, and when the called video image of the calling party is clear, the process goes to step S520.
  • Step S510 the video adjustment module in the calling device can adjust the value of encQuality in the encoder, set the encQuality value to high, and improve the image quality in the video encoding process.
  • Step S512 when the called device returns that the video image of the calling party is not smooth, go to step S514, and when the called video image of the calling party is smooth, go to step S520.
  • step S514 the user can set the macroblock division to 8X8 to reduce the complexity of the encoding.
  • step S516 when the slidability of the video image of the called party is not enough, the process goes to step S518, and when the called party returns the video image of the calling party has been smooth, the process goes to step S520.
  • step S5128 the range of the MV can be adjusted.
  • the range of the MV is high, medium, and low. If the current is the highest level, the outer boundary of the image can be found, and the value can be adjusted to a mid-range. If the image is not smooth enough, you can set the MV range to low.
  • step S518a may be further included: if the called party or the feedback video image is not smooth, the encQuality value may be checked, whether encQuality is high, and if it is high, the value is adjusted to medium or low. .
  • the adjustment of the relevant parameters in steps S510, S514, S518, and S518a is different according to the current registration network of the user, and the range of parameter adjustment is different.
  • the network registered by the user is 4G and the network bandwidth limitation is, for example, the video bandwidth limit of 1280X720@30fps is 4M.
  • the algorithm of the Baseline specification selected by the user can set the value of encQuality to medium or low. The finest macroblock division can be adjusted between 8X8 and 16X16, and the MV range is adjusted from low to medium.
  • embodiments of the present invention may be embodied in the form of a software product stored in a storage medium (eg, ROM/RAM, disk, optical disk) including a number of instructions for causing a terminal device (may be a cell phone, computer, server, or network device, etc.) performs the methods described in various embodiments of the present invention.
  • a storage medium eg, ROM/RAM, disk, optical disk
  • a terminal device may be a cell phone, computer, server, or network device, etc.
  • an image processing device is further provided, which is used to implement the above-mentioned embodiments and optional embodiments, and has not been described again.
  • the term “module” may implement a combination of software and/or hardware of a predetermined function. Although described in the following embodiments The device may be implemented in software, but hardware, or a combination of software and hardware, is also possible and contemplated.
  • FIG. 6 is a structural block diagram of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 6, the apparatus includes:
  • the sending module 62 is configured to send a video image to the called device
  • the receiving module 64 is connected to the sending module 62, and is configured to receive a notification message fed back by the called device according to the video image, wherein the notification message carries an image adjustment parameter of the video image, where the image adjustment parameter includes a sharpness parameter and a video image. At least one of the fluency parameters;
  • the adjustment module 66 is connected to the receiving module 64, and is configured to adjust the video image during the video call according to the image adjustment parameter, for example, adjusting the sharpness of the video image during the video call according to the sharpness parameter, and adjusting according to the fluency parameter The fluency of the video image during a video call.
  • the adjusting module 66 is configured to determine, when the image adjustment parameter includes the sharpness parameter, whether the sharpness of the video image reaches a first predetermined value during the video call according to the sharpness parameter; and when the image adjustment parameter includes the smoothness When the parameter is used, it is determined according to the fluency parameter whether the fluency of the video image during the video call reaches a second predetermined value; and in the case that the determination result is no, correspondingly adjusting the clarity and fluency of the video image during the video call At least one.
  • the adjustment module 66 is configured to adjust a predetermined adjustment parameter for controlling the sharpness of the video image; and adjust the sharpness of the video image during the video call according to the adjusted predetermined adjustment parameter.
  • the adjusting module 66 is configured to adjust the fluency of the video image during the video call by adjusting a macroblock division of the video image, where one macroblock includes one luma pixel block and two additional chrominances. Pixel block.
  • the adjusting module 66 is further configured to: after adjusting the trajectory of the video image during the video call by adjusting the macroblock division manner of the video image, determining whether the fluency of the adjusted video image reaches a second predetermined value; And in the case that the determination result is no, the fluency of the video image during the video call is adjusted by adjusting the range of the motion vector MV, wherein the MV is the current coded macroblock and the reference frame of the video image during inter prediction. The most matching degree with the current coded macroblock Motion vector between high macroblocks.
  • each of the foregoing modules may be implemented by software or hardware.
  • the foregoing may be implemented by, but not limited to, the foregoing modules are all located in the same processor; or, each of the above modules is The form of any combination is located in a different processor.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be configured to store program code for performing the following steps:
  • Adjust a video image during a video call according to the image adjustment parameter such as adjusting a sharpness of a video image during a video call according to a sharpness parameter, and adjusting a smoothness of the video image during the video call according to the fluency parameter.
  • the storage medium is further configured to store program code for performing the following steps: adjusting the sharpness of the video image during the video call includes:
  • the storage medium is further configured to store program code for performing the following steps: adjusting the fluency of the video image during the video call includes:
  • the storage medium is further configured to store program code for performing the following steps: after adjusting the fluency of the video image during the video call by adjusting a macroblock division manner of the video image, the method further includes:
  • the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • a mobile hard disk e.g., a hard disk
  • magnetic memory e.g., a hard disk
  • the processor is configured to: send a video image to the called device according to the stored program code in the storage medium; receive a notification message that is sent by the called device according to the video image, where the notification message carries the video.
  • An image adjustment parameter of the image the image adjustment parameter including at least one of a sharpness parameter and a fluency parameter of the video image; the image adjustment parameter adjusting a video image during a video call, such as adjusting a video call process according to the sharpness parameter
  • the sharpness of the video image in the video, and the smoothness of the video image during the video call is adjusted according to the fluency parameter.
  • the processor is executed according to the stored program code in the storage medium: adjusting the video image during the video call according to the image adjustment parameter includes: when the image adjustment parameter includes the sharpness parameter, according to The sharpness parameter determines whether the sharpness of the video image reaches a first predetermined value during the video call; when the image adjustment parameter includes the fluency parameter, determines whether the smoothness of the video image during the video call reaches a second predetermined value according to the fluency parameter In the case where the determination result is no, at least one of the definition and the smoothness of the video image during the video call is adjusted correspondingly.
  • adjusting the definition of the video image during the video call includes: adjusting the clarity of the video image for controlling The predetermined adjustment parameter of the degree; adjusting the definition of the video image during the video call according to the adjusted predetermined adjustment parameter.
  • the processor performs, according to the stored program code in the storage medium, adjusting the fluency of the video image during the video call, including: adjusting the video call process by adjusting a macroblock division manner of the video image.
  • the smoothness of the video image wherein one macroblock includes one luma pixel block and two additional chroma pixel blocks.
  • the processor performs, according to the stored program code in the storage medium, after adjusting the trajectory of the video image during the video call by adjusting a macroblock division manner of the video image, the method further includes: determining Whether the fluency of the adjusted video image reaches a second predetermined value; if the determination result is no, the fluency of the video image during the video call is adjusted by adjusting the range of the motion vector MV, wherein the MV is a frame The inter-prediction, the motion vector between the currently coded macroblock of the video image and the macroblock in the reference frame that has the highest degree of matching with the currently coded macroblock.
  • Computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, or other data. , removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage, A magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be used to store desired information and that can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media.
  • the above embodiment can at least adjust the sharpness or smoothness of the video image during the video call.

Abstract

一种图像处理方法及装置，该方法包括：向被叫设备发送视频图像(S202)；接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和视频图像的流畅度参数中的至少一种(S204)；根据所述图像调整参数调整视频通话过程中的视频图像(S206) 。

Description

图像处理方法及装置 技术领域
本文涉及通信领域，具体而言，涉及一种图像处理方法及装置。
背景技术
随着智能终端(例如，智能手机)的普及和移动互联网业4G长期演进(Long Term Evolution，简称为LTE)、IP多媒体子系统(IP Multimedia Subsystem，简称为IMS)、远程控制系统(Remote Control System，简称为RCS)技术的迅猛发展，视频通话业务也顺利成章成为基本业务被人们普遍使用，视频通话过程中画质的清晰和流畅是用户的基本要求。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本发明实施例提供了一种图像处理方法及装置，能够至少调节视频通话过程中视频图像的清晰度和流畅度中至少一种。
根据本发明的一个实施例，提供了一种图像处理方法，包括：向被叫设备发送视频图像；接收被叫设备根据所述视频图像反馈的通知消息，其中，所述通知消息携带有所述视频图像的图像调整参数，所述图像调整参数包括清晰度参数和流畅度参数中至少一种；根据所述图像调整参数调整视频通话过程中的视频图像。
可选地，根据所述图像调整参数调整视频通话过程中的视频图像包括：当通知消息携带有视频图像的清晰度参数时，根据所述清晰度参数判断所述视频通话过程中视频图像的清晰度是否达到第一预定值；当通知消息携带有视频图像的流畅度参数时，根据所述流畅度参数判断所述视频通话过程中视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，对应调整 所述视频通话过程中视频图像的清晰度和流畅度中至少一种。
可选地，当判断为调整所述视频通话过程中视频图像的清晰度时，所述调整所述视频通话过程中视频图像的清晰度包括：调整用于控制所述视频图像的清晰度的预定调整参数；根据所述调整后的预定调整参数调整所述视频通话过程中视频图像的清晰度。
可选地，当判断为调整所述视频通话过程中视频图像的流畅度时，所述调整所述视频通话过程中视频图像的流畅度包括：通过调整所述视频图像的宏块划分的方式，调整所述视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，在通过调整所述视频图像的宏块划分方式，调整所述视频通话过程中视频图像的流畅度之后，还包括：判断调整后的视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整所述视频通话过程中视频图像的流畅度，其中，所述MV是帧间预测时，所述视频图像的当前编码宏块和参考帧中与所述当前编码宏块匹配度最高的宏块间的运动矢量。
根据本发明的另一个实施例，提供了一种图像处理装置，包括：发送模块，设置为向被叫设备发送视频图像；接收模块，设置为接收被叫设备根据所述视频图像反馈的通知消息，其中，所述通知消息携带有所述视频图像的图像调整参数，所述图像调整参数包括清晰度参数和所述视频图像的流畅度参数中至少一种；调整模块，设置为根据所述图像调整参数调整视频通话过程中视频图像。
可选地，所述调整模块，是设置为当图像调整参数中包括清晰度参数时，根据所述清晰度参数判断所述视频通话过程中视频图像的清晰度是否达到第一预定值；当图像调整参数包括流畅度参数时，根据所述流畅度参数判断所述视频通话过程中视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，对应调整所述视频通话过程中视频图像的清晰度和流畅度中至少一种。
可选地，所述调整模块，是设置为当判断为调整所述视频通话过程中视频图像的清晰度时，通过如下方式调整所述视频通话过程中视频图像的清晰 度：调整用于控制所述视频图像的清晰度的预定调整参数；以及根据所述调整后的预定调整参数调整所述视频通话过程中视频图像的清晰度。
可选地，所述调整模块，是设置为当判断为调整所述视频通话过程中视频图像的流畅度时，通过如下方式调整所述视频通话过程中视频图像的流畅度：通过调整所述视频图像的宏块划分的方式，调整所述视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，所述调整模块，还设置为在通过调整所述视频图像的宏块划分方式，调整所述视频通话过程中视频图像的流畅度之后，判断调整后的视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整所述视频通话过程中视频图像的流畅度，其中，所述MV是帧间预测时，所述视频图像的当前编码宏块和参考帧中与所述当前编码宏块匹配度最高的宏块间的运动矢量。
根据本发明的又一个实施例，还提供了一种存储介质。该存储介质设置为存储用于执行以下步骤的程序代码：向被叫设备发送视频图像；接收被叫设备根据所述视频图像反馈的通知消息，其中，所述通知消息携带有所述视频图像的图像调整参数，所述图像调整参数包括清晰度参数和流畅度参数中至少一种；根据所述图像调整参数调整视频通话过程中的视频图像。
可选地，存储介质还设置为存储用于执行以下步骤的程序代码：根据所述图像调整参数调整视频通话过程中的视频图像包括：当通知消息携带有视频图像的清晰度参数时，根据所述清晰度参数判断所述视频通话过程中视频图像的清晰度是否达到第一预定值；当通知消息携带有视频图像的流畅度参数时，根据所述流畅度参数判断所述视频通话过程中视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，对应调整所述视频通话过程中视频图像的清晰度和流畅度中至少一种。
可选地，存储介质还设置为存储用于执行以下步骤的程序代码：当判断为调整所述视频通话过程中视频图像的清晰度时，所述调整所述视频通话过程中视频图像的清晰度包括：调整用于控制所述视频图像的清晰度的预定调整参数；根据所述调整后的预定调整参数调整所述视频通话过程中视频图像 的清晰度。
可选地，存储介质还设置为存储用于执行以下步骤的程序代码：当判断为调整所述视频通话过程中视频图像的流畅度时，所述调整所述视频通话过程中视频图像的流畅度包括：通过调整所述视频图像的宏块划分的方式，调整所述视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，存储介质还设置为存储用于执行以下步骤的程序代码：在通过调整所述视频图像的宏块划分方式，调整所述视频通话过程中视频图像的流畅度之后，还包括：判断调整后的视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整所述视频通话过程中视频图像的流畅度，其中，所述MV是帧间预测时，所述视频图像的当前编码宏块和参考帧中与所述当前编码宏块匹配度最高的宏块间的运动矢量。
通过本发明实施例，向被叫设备发送视频图像；接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和流畅度参数中至少一种；根据所述图像调整参数调整视频通话过程中的视频图像。由于通过被叫设备发送的视频图像的清晰度参数和视频图像的流畅度参数中的至少一种对应调整视频通话过程中视频图像的清晰度和流畅度中至少一种。因此，本发明实施例可以至少调节视频通话过程中视频图像的清晰度和流畅度中至少一种。
附图概述
图1是本发明实施例的一种图像处理方法的移动终端的硬件结构框图；
图2是根据本发明实施例的图像处理方法的流程图；
图3是根据本发明实施例的基于H264编码的视频通话移动终端示意图；
图4是根据本发明实施例H264宏块和子块的划分示意图；
图5是根据本发明实施例的H264编码器视频通话流程图；
图6是根据本发明实施例的图像处理装置的结构框图。
本发明的实施方式
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
视频通话过程中采用的算法大部分是H264算法，H.264引入的新技术带来了较高的压缩比，同时大大提高了算法的复杂度。但H264协议有分层设计的概念，不同场景的视频编码器可以采用不同的技术算法。对于应用于简单的场景采用baseline profile算法，对于应用于复杂的场景采用main profile或者是high profile算法。受终端处理器及相关硬件的限制，终端上使用的H264编码器一直是baseline profile规格的编码算法，但随着终端的处理器及相关硬件技术的发展，终端的视频编码器将不局限于基于baseline Profile的算法，有可能会采用main Profile规格的编码算法。
但是，无论采用baseline profile算法，还是采用main profile或者high profile算法，均无法调节视频通话过程中视频图像的清晰度或流畅度。
本申请方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例，图1是本发明实施例的一种图像处理方法的移动终端的硬件结构框图。如图1所示，移动终端10可以包括：一个或多个(图中仅示出一个)处理器102(处理器102可以包括微处理器MCU或可编程逻辑器件FPGA等的处理装置)、设置为存储数据的存储器104、以及设置为通信功能的传输装置106。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述电子装置的结构造成限定。例如，移动终端10还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。
存储器104可设置为存储应用软件的软件程序以及模块，如本发明实施 例中的图像处理方法对应的程序指令或模块，处理器102设置为通过运行存储在存储器104内的软件程序以及模块，从而执行多种功能应用以及数据处理，即实现上述的方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可进一步包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至移动终端10。上述网络的实例包括互联网、企业内部网、局域网、移动通信网及其组合。
传输装置106设置为经由一个网络接收或者发送数据。上述的网络实例可包括移动终端10的通信供应商提供的无线网络。在一个实例中，传输装置106包括一个网络适配器(Network Interface Controller，NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置106可以为射频(Radio Frequency，RF)模块，其设置为通过无线方式与互联网进行通讯。
在本实施例中提供了一种运行于上述移动终端的图像处理方法，图2是根据本发明实施例的图像处理方法的流程图，如图2所示，该流程包括如下步骤：
步骤S202，向被叫设备发送视频图像；
步骤S204，接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和视频图像的流畅度参数中的至少一种；
步骤S206，根据所述图像调整参数调整视频通话过程中的视频图像，如根据清晰度参数调整视频通话过程中的视频图像的清晰度，根据流畅度参数调整视频通话过程中的视频图像的流畅度。
上述步骤，通过被叫设备发送的视频图像的清晰度参数和视频图像的流畅度参数中至少一种对应调整视频通话过程中视频图像的清晰度和流畅度中至少一种。
可选地，根据所述图像调整参数调整视频通话过程中的视频图像包括：当通知消息携带有视频图像的清晰度参数时，根据清晰度参数判断视频通话 过程中视频图像的清晰度是否达到第一预定值；当通知消息携带有视频图像的流畅度参数时，根据流畅度参数判断视频通话过程中视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，对应调整视频通话过程中视频图像的清晰度和流畅度中的至少一种。
可选地，调整视频通话过程中视频图像的清晰度包括：调整用于控制视频图像的清晰度的预定调整参数；根据调整后的预定调整参数调整视频通话过程中视频图像的清晰度。
可选地，调整视频通话过程中视频图像的流畅度包括：通过调整视频图像的宏块划分的方式，调整视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，在通过调整视频图像的宏块划分方式，调整视频通话过程中视频图像的流畅度之后，还包括：判断调整后的视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，通过调整运动矢量(Motion Vector，简称为MV)的范围的方式，调整视频通话过程中视频图像的流畅度，其中，MV是帧间预测时，视频图像的当前编码宏块和参考帧中与当前编码宏块匹配度最高的宏块间的运动矢量。
为了方便理解上述实施例，本发明实施例提供了一种基于H264编码的视频通话移动终端，通过本发明实施例，用户在移动终端上进行视频通话过程中，可以保证清晰流畅的画质，另外如果画面不清晰，用户可以手动调解视频质量和画面流畅性。图3是根据本发明实施例的基于H264编码的视频通话移动终端示意图，如图3所示，该基于H264编码的视频通话移动终端包括：视频电话模块、参数配置模块，视频调节模块、网络制式监控模块、H264编码器模块。
视频电话模块，设置为当发起或者接听视频电话时，向底层传递一个视频呼叫的指示标志，所述指示标志指示当前的场景是视频通话场景。
参数配置模块，设置为保存编码器在编码的开始和过程中需设的参数值。参数值可以分为三种规格算法的参数值，分别是Baseline Profile、Main Profile、High Profile三种规格算法的参数值。每种规格算法又包括Base Params和Dynamic Params参数组成。Base Params是编码器创建时需设置的 参数，此部分参数值一般会设成最小参考值。Dynamic Params是编码器运行的过程中，根据视频源的变化动态进行的设置的参数，此参数在每一帧开始编码前进行设置。当上层传递的指示标志显示当前用户正在发起或者接听视频通话，则启动视频编码器，并选择Base line的参数值。
Baseline Profile参数
{
I and P slice
Multiple Reference Frames
In-loop Deblocking Fliter
CAVLC Entropy Coding
}
Base Params参数
{
Size
encodingPreset
rateControlPreset
maxHeight
maxWidth
maxFrameRate
maxBitRate
…
}
Dynamic Params参数
{
rateControlPreset
maxFrameRate
maxBitRate
MV
Range MV
}
视频调节模块，设置为根据被叫方反馈的视频画面质量和流畅性，调节编码器的参数，实时调节视频通话过程中的视频质量和流畅性。调节参数包括调节encodingPreset参数，调节宏块的划分方式，调节MV(Motion vector)的范围。
encodingPreset参数用于控制视频通话过程中的画面质量，可以分高、中、低。当选择高时，视频编码清晰度会高，相应的码流数据会大，压缩率会低。
通过调节宏块划分方式来调节视频流畅度。H264协议较之前的H263和MPEG4协议，压缩率提升了，算法复杂度也提升了，其中最重要的一项改进是宏块划分最小可以细化到4X4，而H263宏块划分最细为8X8。H.264宏块有多种类型，并引入了子块的概念，H264有7种宏块划分模式，可以按照16X16，16X8，8X16，8X8进行分割：如果选择8X8的宏块模式，可以按照8X4，4X8，4X4进行子宏块分割。宏块和子块的划分如图4所示。
H.264宏块和子块的划分：编码时采用什么类型的宏块划分，可以通过试算法，对每个宏块划分成不同种类型，然后算出绝对误差和值(Sum Absolute Difference，简称为SAD)，最后选出最优方式进行宏块的划分。由于H264宏块划分类型多样，每种算法都遍历一遍，然后找到最优划分方法，这样耗时，不利用视频的流畅性。视频通话过程中，画面一般比较简单，主体主要是人物，另外画风变化一般不会太大，变化最多的是眼睛和嘴巴，另外根据对视频通话过程中码流进行分析，视频通话过程中80％的宏块划分都是1X1,只有9％有可能划分成16X16。所以视频编码过程中可以通过调节宏块划分类型，来提升或者减缓编码的速度，而编码速度体现在视频的流畅度。
通过调节运动矢量(Motion Vector，简称为MV)的范围来调节视频的 流畅度，MV是帧间预测时当前编码宏块和参考帧中匹配度最高宏块间的运动矢量。此匹配度最高的宏块搜索方法，可以在参考帧中一个宏块一个宏块匹配计算，然后找出最匹配的宏块，这样一个一个适配计算，影响编码的速度。如果在编码过程中，在宏块预测时定一个搜索范围，在定的范围内搜匹配的宏块，可以减少适配时间，有利于提升编码速度。可以把MV的范围设为几个档位。对于视频通话，根据画面特点，如上解释，视频通话过程画风变化简单，所以每个宏块不需要对整个参考帧进行匹配，可能在很小的范围就可以找到匹配的宏块，所以在编码时可以把MV的范围设为稍低档位，减小匹配宏块的搜索范围，减少编码时间，提升编码速度，提升视频流畅度。
网络制式监控模块，设置为监控当前用户注册的网络制式，然后反馈给视频调节模块。因网络制式不同，网络对视频带宽限制不同，则调节视频通话的参数(encodingPreset，宏块划分方式，MV范围)区间不同。
H264编码器，设置为根据参数配置模块和用户实时调节的配置参数对前摄传进来的图像数据进行编码。
图5是根据本发明实施例的H264编码器视频通话流程图，如图5所示，该流程包括：
步骤S502，用户发起或者接听视频通话，给底层传递一个视频通话的指示标志，指示当前的场景是视频通话场景。
步骤S504，编码器配置模块收到上层传递视频指示标志，选择配置模块中的Baseline规格算法。
步骤S506，Baseline规格算法确定后，配置编码器初始值，编码器初始值为Base Params，此部分参数可以设为最小值。Dynamic Params是编码器运行的过程中，根据视频源的变化动态进行设置的参数，此参数需要每一帧开始编码前进行设置。
步骤S508，当被叫反馈主叫方的视频图像不清晰时，转到步骤S510，当被叫反馈主叫方的视频图像清晰时，转到步骤S520。
步骤S510，主叫设备中视频调节模块可以调节编码器中encQuality的值，把encQuality值设为高，提高视频编码过程中的图像质量。
步骤S512，当被叫设备反馈主叫方的视频图像不流畅时，转到步骤S514，当被叫反馈主叫方的视频图像流畅时，转到步骤S520。
步骤S514，用户可以把宏块划分最细设为8X8，减小编码的复杂度。
步骤S516，当被叫反馈主叫方的视频图像的流畅度还不够，转到步骤S518，当被叫反馈主叫方的视频图像已流畅，转到步骤S520。
步骤S518，可以调节MV的范围，MV的范围是高、中、低三档，如果当前是最高档，可以搜到图像的最边界，可以把此值调成中档。如果图像还不够流畅，可以把MV范围设为低档。
步骤S520，结束。
可选地，在步骤S518之后，还可以包括步骤S518a：如果被叫方还是反馈视频图像不流畅时，可以检查encQuality值，encQuality是否为高，如果是高，则把此值调为中或者低。
另外在步骤S510、S514、S518，S518a中相关参数的调节根据用户当前注册网络的不同，参数调节的范围不同。当用户注册的网络是4G，网络对视频带宽限制，例如1280X720@30fps时的视频带宽限制是4M，此时用户选择的Baseline规格的算法，则encQuality的值可以设为中或者低。宏块划分最细可以在8X8到16X16之间调节，MV的范围最大在低到中之间调节。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明实施例可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
在本实施例中还提供了一种图像处理装置，该装置用于实现上述实施例及可选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的 装置可以地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图6是根据本发明实施例的图像处理装置的结构框图，如图6所示，该装置包括：
发送模块62，设置为向被叫设备发送视频图像；
接收模块64，连接至上述发送模块62，设置为接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和视频图像的流畅度参数中的至少一种；
调整模块66，连接至上述接收模块64，设置为根据所述图像调整参数调整视频通话过程中视频图像，如，根据清晰度参数调整视频通话过程中的视频图像的清晰度，根据流畅度参数调整视频通话过程中的视频图像的流畅度。
可选地，调整模块66，是设置为当图像调整参数中包括清晰度参数时，根据清晰度参数判断视频通话过程中视频图像的清晰度是否达到第一预定值；当图像调整参数包括流畅度参数时，根据流畅度参数判断视频通话过程中视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，对应调整视频通话过程中视频图像的清晰度和流畅度中的至少一种。
可选地，调整模块66，是设置为调整用于控制视频图像的清晰度的预定调整参数；以及根据调整后的预定调整参数调整视频通话过程中视频图像的清晰度。
可选地，调整模块66，是设置为通过调整视频图像的宏块划分的方式，调整视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，调整模块66，还设置为在通过调整视频图像的宏块划分方式，调整视频通话过程中视频图像的流畅度之后，判断调整后的视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整视频通话过程中视频图像的流畅度，其中，MV是帧间预测时，视频图像的当前编码宏块和参考帧中与当前编码宏块匹配度最 高的宏块间的运动矢量。
需要说明的是，上述每个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述每个模块以任意组合的形式分别位于不同的处理器中。
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以被设置为存储用于执行以下步骤的程序代码：
S1，向被叫设备发送视频图像；
S2，接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和视频图像的流畅度参数中的至少一种；
S3，根据所述图像调整参数调整视频通话过程中的视频图像，如根据清晰度参数调整视频通话过程中的视频图像的清晰度，根据流畅度参数调整视频通话过程中的视频图像的流畅度。
可选地，存储介质还被设置为存储用于执行以下步骤的程序代码：根据所述图像调整参数调整视频通话过程中的视频图像包括：
S1，当通知消息携带有视频图像的清晰度参数时，根据清晰度参数判断视频通话过程中视频图像的清晰度是否达到第一预定值；当通知消息携带有视频图像的流畅度参数时，根据流畅度参数判断视频通话过程中视频图像的流畅度是否达到第二预定值；
S2，在判断结果为否的情况下，对应调整视频通话过程中视频图像的清晰度和流畅度中的至少一种。
可选地，存储介质还被设置为存储用于执行以下步骤的程序代码：调整视频通话过程中视频图像的清晰度包括：
S1，调整用于控制视频图像的清晰度的预定调整参数；
S2，根据调整后的预定调整参数调整视频通话过程中视频图像的清晰度。
可选地，存储介质还被设置为存储用于执行以下步骤的程序代码：调整视频通话过程中视频图像的流畅度包括：
S1，通过调整视频图像的宏块划分的方式，调整视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，存储介质还被设置为存储用于执行以下步骤的程序代码：在通过调整视频图像的宏块划分方式，调整视频通话过程中视频图像的流畅度之后，还包括：
S1，判断调整后的视频图像的流畅度是否达到第二预定值；
S2，在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整视频通话过程中视频图像的流畅度，其中，MV是帧间预测时，视频图像的当前编码宏块和参考帧中与当前编码宏块匹配度最高的宏块间的运动矢量。
可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等多种可以存储程序代码的介质。
可选地，在本实施例中，处理器根据存储介质中已存储的程序代码执行：向被叫设备发送视频图像；接收被叫设备根据视频图像反馈的通知消息，其中，通知消息携带有视频图像的图像调整参数，所述图像调整参数包括清晰度参数和视频图像的流畅度参数中的至少一种；所述图像调整参数调整视频通话过程中视频图像，如根据清晰度参数调整视频通话过程中的视频图像的清晰度，根据流畅度参数调整视频通话过程中的视频图像的流畅度。
可选地，在本实施例中，处理器根据存储介质中已存储的程序代码执行：根据所述图像调整参数调整视频通话过程中视频图像包括：当图像调整参数中包括清晰度参数时，根据清晰度参数判断视频通话过程中视频图像的清晰度是否达到第一预定值；当图像调整参数包括流畅度参数时，根据流畅度参数判断视频通话过程中视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，对应调整视频通话过程中视频图像的清晰度和流畅度中的至少一种。
可选地，在本实施例中，处理器根据存储介质中已存储的程序代码执行：调整视频通话过程中视频图像的清晰度包括：调整用于控制视频图像的清晰 度的预定调整参数；根据调整后的预定调整参数调整视频通话过程中视频图像的清晰度。
可选地，在本实施例中，处理器根据存储介质中已存储的程序代码执行：调整视频通话过程中视频图像的流畅度包括：通过调整视频图像的宏块划分的方式，调整视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
可选地，在本实施例中，处理器根据存储介质中已存储的程序代码执行：在通过调整视频图像的宏块划分方式，调整视频通话过程中视频图像的流畅度之后，还包括：判断调整后的视频图像的流畅度是否达到第二预定值；在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整视频通话过程中视频图像的流畅度，其中，MV是帧间预测时，视频图像的当前编码宏块和参考帧中与当前编码宏块匹配度最高的宏块间的运动矢量。
可选地，本实施例中的示例可以参考上述实施例及可选实施方式中所描述的示例，本实施例在此不再赘述。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理单元的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、 磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。以上所述仅为本发明的可选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
工业实用性
上述实施例可以至少调节视频通话过程中视频图像的清晰度或流畅度。

Claims (11)

  1. 一种图像处理方法，包括：
    向被叫设备发送视频图像(S202)；
    接收被叫设备根据所述视频图像反馈的通知消息，其中，所述通知消息携带有所述视频图像的图像调整参数，所述图像调整参数包括清晰度参数和所述视频图像的流畅度参数中的至少一种(S204)；
    根据所述图像调整参数调整视频通话过程中的视频图像(S206)。
  2. 根据权利要求1所述的方法，其中，所述根据所述图像调整参数调整视频通话过程中的视频图像(S206)包括：
    当通知消息携带有视频图像的清晰度参数时，根据所述清晰度参数判断所述视频通话过程中视频图像的清晰度是否达到第一预定值；当通知消息携带有视频图像的流畅度参数时，根据所述流畅度参数判断所述视频通话过程中视频图像的流畅度是否达到第二预定值；
    在判断结果为否的情况下，对应调整所述视频通话过程中视频图像的清晰度和流畅度中的至少一种。
  3. 根据权利要求2所述的方法，其中，当判断为调整所述视频通话过程中视频图像的清晰度时，所述调整所述视频通话过程中视频图像的清晰度包括：
    调整用于控制所述视频图像的清晰度的预定调整参数；
    根据所述调整后的预定调整参数调整所述视频通话过程中视频图像的清晰度。
  4. 根据权利要求2所述的方法，其中，当判断为调整所述视频通话过程中视频图像的流畅度时，所述调整所述视频通话过程中视频图像的流畅度包括：
    通过调整所述视频图像的宏块划分的方式，调整所述视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
  5. 根据权利要求4所述的方法，还包括：
    在通过调整所述视频图像的宏块划分方式，调整所述视频通话过程中视频图像的流畅度之后，判断调整后的视频图像的流畅度是否达到第二预定值；
    在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整所述视频通话过程中视频图像的流畅度，其中，所述MV是帧间预测时，所述视频图像的当前编码宏块和参考帧中与所述当前编码宏块匹配度最高的宏块间的运动矢量。
  6. 一种图像处理装置，包括：
    发送模块(62)，设置为向被叫设备发送视频图像；
    接收模块(64)，设置为接收被叫设备根据所述视频图像反馈的通知消息，其中，所述通知消息携带有所述视频图像的图像调整参数，所述图像调整参数包括清晰度参数和所述视频图像的流畅度参数中的至少一种；
    调整模块(66)，设置为根据所述图像调整参数调整视频通话过程中视频图像。
  7. 根据权利要求6所述的装置，其中，所述调整模块(66)，是设置为当图像调整参数中包括清晰度参数时，根据所述清晰度参数判断所述视频通话过程中视频图像的清晰度是否达到第一预定值；当图像调整参数包括流畅度参数时，根据所述流畅度参数判断所述视频通话过程中视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，对应调整所述视频通话过程中视频图像的清晰度和流畅度中的至少一种。
  8. 根据权利要求7所述的装置，其中，所述调整模块(66)，是设置为当判断为调整所述视频通话过程中视频图像的清晰度时，通过如下方式调整所述视频通话过程中视频图像的清晰度：
    调整用于控制所述视频图像的清晰度的预定调整参数；以及根据所述调整后的预定调整参数调整所述视频通话过程中视频图像的清晰度。
  9. 根据权利要求7所述的装置，其中，所述调整模块(66)，是设置为当判断为调整所述视频通话过程中视频图像的流畅度时，通过如下方式调 整所述视频通话过程中视频图像的流畅度：
    通过调整所述视频图像的宏块划分的方式，调整所述视频通话过程中视频图像的流畅度，其中，一个宏块包括一个亮度像素块和附加的两个色度像素块。
  10. 根据权利要求9所述的装置，所述调整模块(66)，还设置为在通过调整所述视频图像的宏块划分方式，调整所述视频通话过程中视频图像的流畅度之后，判断调整后的视频图像的流畅度是否达到第二预定值；以及在判断结果为否的情况下，通过调整运动矢量MV的范围的方式，调整所述视频通话过程中视频图像的流畅度，其中，所述MV是帧间预测时，所述视频图像的当前编码宏块和参考帧中与所述当前编码宏块匹配度最高的宏块间的运动矢量。
  11. 一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现权利要求1至5中任一项所述的方法。
PCT/CN2017/106740 2016-10-20 2017-10-18 图像处理方法及装置 WO2018072716A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610913618.6A CN107968927A (zh) 2016-10-20 2016-10-20 图像处理方法及装置
CN201610913618.6 2016-10-20

Publications (1)

Publication Number Publication Date
WO2018072716A1 true WO2018072716A1 (zh) 2018-04-26

Family

ID=61996290

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/106740 WO2018072716A1 (zh) 2016-10-20 2017-10-18 图像处理方法及装置

Country Status (2)

Country Link
CN (1) CN107968927A (zh)
WO (1) WO2018072716A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070189275A1 (en) * 2006-02-10 2007-08-16 Ralph Neff System and method for connecting mobile devices
CN101998102A (zh) * 2009-08-24 2011-03-30 中兴通讯股份有限公司 一种控制移动可视电话视频质量的方法和移动可视电话
CN104253967A (zh) * 2014-09-26 2014-12-31 厦门亿联网络技术股份有限公司 一种实时视频通信传输控制方法
CN105812706A (zh) * 2016-03-17 2016-07-27 掌赢信息科技（上海）有限公司 一种视频通话质量评估方法及电子设备
CN105812711A (zh) * 2016-05-05 2016-07-27 广东小天才科技有限公司 视频通话过程中优化图像质量的方法及系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013059016A (ja) * 2011-08-12 2013-03-28 Sony Corp 画像処理装置および方法、並びにプログラム
CN104796706A (zh) * 2014-01-17 2015-07-22 深圳市中瀛鑫科技股份有限公司 一种视频编码方法及装置
US10091511B2 (en) * 2015-01-05 2018-10-02 Getgo, Inc. Efficient video block matching

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070189275A1 (en) * 2006-02-10 2007-08-16 Ralph Neff System and method for connecting mobile devices
CN101998102A (zh) * 2009-08-24 2011-03-30 中兴通讯股份有限公司 一种控制移动可视电话视频质量的方法和移动可视电话
CN104253967A (zh) * 2014-09-26 2014-12-31 厦门亿联网络技术股份有限公司 一种实时视频通信传输控制方法
CN105812706A (zh) * 2016-03-17 2016-07-27 掌赢信息科技（上海）有限公司 一种视频通话质量评估方法及电子设备
CN105812711A (zh) * 2016-05-05 2016-07-27 广东小天才科技有限公司 视频通话过程中优化图像质量的方法及系统

Also Published As

Publication number Publication date
CN107968927A (zh) 2018-04-27

Similar Documents

Publication Publication Date Title
JP6359764B2 (ja) 対話型ビデオ会議
TWI829058B (zh) 用於在多媒體通訊中使用壓縮並行轉碼器的方法和裝置
US11025933B2 (en) Dynamic video configurations
CN110572684A (zh) 扩展四叉树、不等四划分的主要概念以及信令
US11431997B2 (en) Video decoding method and video decoder
KR20170084114A (ko) 비디오 코딩에서 인트라 예측을 위한 시스템 및 방법
CN111294590A (zh) 用于多假设编码的加权预测方法及装置
US11895297B2 (en) Prediction mode determining method and apparatus, encoding device, and decoding device
DE102013101817A1 (de) Verfahren zum bereitstellen einer kommunikationssitzung und vorrichtung
WO2018161867A1 (zh) 码率分配方法、设备及存储介质
US20170006078A1 (en) Methods and apparatus for codec negotiation in decentralized multimedia conferences
US20190109884A1 (en) Control Of Media Transcoding During A Media Session
WO2021185257A1 (zh) 图像编码方法、图像解码方法及相关装置
JP2018529249A (ja) ビデオ電話におけるディスプレイデバイスを切り替えること
TW202127874A (zh) 用於視訊編碼中的自我調整色彩變換的qp 偏移的靈活訊號傳遞
WO2018072716A1 (zh) 图像处理方法及装置
US20170223079A1 (en) ROI Video Implementation Method and Apparatus
US20220408093A1 (en) Video decoding method and device for coding chroma quantization parameter offset-related information
US11949858B2 (en) Video throughput improvement using long term referencing, deep learning, and load balancing
WO2021236400A1 (en) Signaling of syntax elements in video coding
EP3893505A1 (en) Method and device for determining prediction mode, coding device, and decoding device
CN111479111B (zh) 图像显示顺序的确定方法、装置和视频编解码设备
WO2023112007A1 (en) Video codec importance indication and radio access network awareness configuration
CN117715685A (zh) 流式传输应用程序中视频数据的微调
WO2020064729A1 (en) Transformation selection by transmitting a transformation set indicator for use in video coding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17861891

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17861891

Country of ref document: EP

Kind code of ref document: A1