WO2022148180A1 - 图像输出方法、装置、计算机设备及计算机可读存储介质 - Google Patents

图像输出方法、装置、计算机设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2022148180A1
WO2022148180A1 PCT/CN2021/134143 CN2021134143W WO2022148180A1 WO 2022148180 A1 WO2022148180 A1 WO 2022148180A1 CN 2021134143 W CN2021134143 W CN 2021134143W WO 2022148180 A1 WO2022148180 A1 WO 2022148180A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
change
image output
output control
threshold
Prior art date
Application number
PCT/CN2021/134143
Other languages
English (en)
French (fr)
Inventor
陈贝
李拓
刘刚
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Priority to US18/259,864 priority Critical patent/US20240320950A1/en
Publication of WO2022148180A1 publication Critical patent/WO2022148180A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20052—Discrete cosine transform [DCT]

Definitions

  • the present application relates to the field of computer technology, and in particular, to an image output method, apparatus, computer device, and computer-readable storage medium.
  • BMC Baseboard Manager Controller, baseboard management controller
  • BMC Baseboard Manager Controller
  • the object of operation is system hardware, such as by monitoring system temperature, voltage, fan, power supply, etc., and doing The corresponding adjustment works to ensure that the system is in a healthy state. It is mainly used to collect various information on a single server and provide it to the upper-layer operation and maintenance network management software through two methods. The first is that BMC will provide various interfaces for the upper-layer network management to query, such as web interface, command line, etc. Human-machine interface; the second is active reporting. When a fault is detected, the BMC can report it to the server of the upper-layer network management software through the network and other means, so that the operation and maintenance personnel can identify and deal with the fault in time.
  • BMC supports KVM (Keyboard Video Mouse, keyboard, video and mouse) over IP service, and builds visualization between local servers and remote managers through KVM services. interface.
  • KVM Keyboard Video Mouse, keyboard, video and mouse
  • the BMC chip uses the network interface to transmit the keyboard, mouse and image data required by the visualization service. It can be understood that if the transmitted image data is large and the network environment is complex, congestion is likely to occur when the image data is directly transmitted through the network, which affects the display delay of the server interface and affects the remote visualization experience. Therefore, the frame data of the image will be compressed and cached in the BMC, and then the compressed image data will be transmitted to the remote through the network.
  • One aspect of the embodiments of the present application provides an image output method, including:
  • the amount of output image data is controlled according to image output control information and network congestion information.
  • generating the image output control information according to the image continuous change feature and the corresponding image change threshold includes:
  • the image output control information is generated according to the image continuous change feature and the corresponding image change threshold, including:
  • the ratio of the high-frequency components to the total number of high- and low-frequency components is not greater than the preset proportion threshold, it is determined that the degree of smooth change in the current frame image is not greater than the preset image change threshold.
  • generating the image output control information according to the image continuous change feature and the corresponding image change threshold includes:
  • generating the image output control information according to the image continuous change feature and the corresponding image change threshold includes:
  • controlling the amount of output image data according to the image output control information and the network congestion information includes:
  • calculating the update value of the quantization step size according to the image output control information and the network congestion information includes:
  • the method before acquiring the image continuous change feature of the display interface of the local server, the method further includes:
  • the step of acquiring the image continuous change feature of the display interface of the local server is performed.
  • an image output device including:
  • the image change information acquisition module is used to acquire the image continuous change feature of the display interface of the local server
  • an image output control information generation module for generating image output control information according to the image continuous change feature and the corresponding image change threshold
  • the image output control module is used for controlling the output image data amount according to the image output control information and the network congestion information.
  • Embodiments of the present application further provide a computer device, including a memory and one or more processors, where computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the one or more processors , causing the one or more processors to execute the steps of any one of the above image output methods.
  • the embodiments of the present application further provide one or more non-volatile computer-readable storage media storing computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, the one or more A plurality of processors perform the steps of any one of the image output methods described above.
  • FIG. 1 is an application scenario diagram of an image output method provided by the present application according to one or more embodiments
  • FIG. 2 is a schematic flowchart of an image output method provided by the present application according to one or more embodiments
  • FIG. 3 is a schematic diagram of a change curve of a first network congestion degree in an illustrative example provided by this application according to one or more embodiments;
  • FIG. 4 is a schematic diagram of a second type of network congestion degree variation curve in an illustrative example provided by this application according to one or more embodiments;
  • FIG. 5 is a schematic diagram of a third variation curve of network congestion degree in an exemplary example provided by this application according to one or more embodiments;
  • FIG. 6 is a schematic diagram of a fourth type of network congestion degree variation curve in an illustrative example provided by this application according to one or more embodiments;
  • FIG. 7 is a schematic diagram of a fifth kind of network congestion degree variation curve in an illustrative example provided by this application according to one or more embodiments;
  • FIG. 8 is a schematic framework diagram of an exemplary application scenario provided by the present application according to one or more embodiments.
  • FIG. 9 is a schematic flowchart of an image change feature determination process provided by the present application according to one or more embodiments.
  • FIG. 10 is a structural diagram of a specific implementation manner of an image output device provided by the present application according to one or more embodiments;
  • FIG. 11 is a schematic diagram of the internal structure of a computer device according to one or more embodiments of the present application.
  • the image output method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the terminal 102a communicates with the local server 104a through the network.
  • the terminal 102a can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server 104a can be implemented by an independent server or a server cluster composed of multiple servers.
  • the terminal 102a obtains the image continuous change feature of the display interface of the local server 104a, generates image output control information according to the image continuous change feature and the corresponding image change threshold, and controls the output image data amount according to the image output control information and the network congestion information.
  • FIG. 2 is a schematic flowchart of an image output method provided by an embodiment of the present application. The method is applied to the terminal in FIG. 1 as an example for description.
  • the embodiment of the present application may include the following content:
  • S101 Acquire the image continuous change feature of the display interface of the local server.
  • the image continuous change feature in this step can be used to represent the change of multiple frames of adjacent images output by the image output end.
  • the degree of color change can also include the degree of content change. The smaller the degree of change, the smaller the difference between adjacent images. Even if one or two frames of images are discarded or the quality of a few of them is not high, it is not easy for users to perceive visually. , the visualization is still good. The greater the degree of change, the greater the difference between adjacent images. If one or two frames of images are discarded or the quality of several of them is not high, the user can easily perceive them, and the visualization effect is not good.
  • the previous frame image is a black and white image
  • the current frame image is an oil painting image, which are two frames of images with large changes
  • the previous frame image is a car image
  • the current frame image is a material metallographic image. It belongs to two frames of images with a large degree of change.
  • the previous frame image is a color car image from a right perspective
  • the current frame image is a color car image from a left perspective, which belong to two frames of images with little change.
  • S102 Generate image output control information according to the image continuous change feature and the corresponding image change threshold.
  • the image output control information can be used to indicate whether to control the amount of image output data and the change characteristics of the currently displayed image, such as whether the output interface is basically unchanged or changing, and the interface image changes drastically or gently.
  • the image output control information may also include other parameters according to the actual application scenario setting. If there are multiple display terminals, the identification information of the corresponding display terminals can be carried in the image output control information at the same time.
  • S103 Control the amount of output image data according to the image output control information and the network congestion information.
  • the network congestion information in this step is used to indicate the amount of image data that the image output terminal is waiting to send and/or about to send.
  • the total output of image data can be reduced by reducing the image quality of some frames, or the total output of image data can be reduced by discarding image data frames and reducing the image quality of some frames at the same time, or other methods can be used to reduce the data at the image output side. total.
  • the continuous change feature of the display interface image of the local server is used as the basis for the BMC to adjust the image encoding process.
  • the data bandwidth requirement at the output end because the data bandwidth requirement at the output end of the network is alleviated based on the quality of the displayed image at the output end, which effectively solves the current situation that the image frame is directly discarded when the network is congested, resulting in poor output image quality. It ensures the quality of the output image and provides users with a better remote visualization experience.
  • FIG. 2 is just a schematic way. , does not mean that it can only be executed in this order.
  • the output control instruction in this step is used to indicate whether to control the output image data, that is, whether to execute the process S101-S103.
  • S101-S103 are executed. For example, for the BMC, when the image output module of the BMC is not congested, S101-S103 can be executed to save the power consumption of the BMC chip and reduce the resource consumption. When the image output module is congested, the workflow of S101-S103 is executed to reduce the data amount of the image output port.
  • step S102 there is no limitation on how to perform step S102.
  • a method for generating image output control information is given, which may include the following steps in conjunction with FIG. 9 :
  • the features of the local server interface image in this embodiment include two types, one is the intensity of changes in luminance and chromaticity within the image, and the other is the degree of approximation between adjacent images.
  • the feature of the local server interface image acquired in step S101 is the image feature that the luminance and chrominance changes in the image are intense, and the intensity of the luminance and chrominance changes in the image indicates the smoothness of the change in the image.
  • a solid-color image has a particularly gentle change in brightness and chroma
  • a color oil painting has a sharper change in brightness and chroma.
  • the implementation process of S102 may be:
  • high and low frequency components can be used to quantitatively evaluate the intensity of changes in luminance and chromaticity within the image or the degree of gentle changes in the image.
  • the "preset image change threshold” process can include:
  • two-dimensional Fourier transform or DCT transform is used to calculate the high and low frequency components of the image, thereby judging the degree of smooth change of the image. If the image changes sharply, the high frequency component of the image is large. The low frequency components are larger.
  • the image cache module can be controlled to change the image quality.
  • the features of the local server interface image acquired in step S101 are image features representing image similarity, and the degree of similarity between adjacent images represents the next interface image How close to the previous interface image. If the interface is still and unchanged, the adjacent images are exactly the same, and even if a few cached image frames are discarded, the interface interaction experience will not be affected. If the frame is cached, the interface will mutate.
  • the implementation process of S102 may be:
  • mutual information can be used to evaluate the similarity of adjacent images, and the corresponding step of "judging whether the similarity between adjacent images is greater than a preset image similarity threshold" may include:
  • the mutual information of the two images is calculated to confirm the degree of similarity of the two images.
  • the larger the mutual information value the greater the degree of similarity of the two images. If the mutual information of the image reaches a certain threshold within a period of time, it is determined that the current server display interface remains basically unchanged. At this time, the method of discarding the cached frame can be used to reduce the data pressure on the image output end, ease the congestion, and at the same time, it will not significantly affect the remote interface display effect.
  • the image cache module can be controlled to discard the image cache frame.
  • step S103 there is no limitation on how to perform step S103.
  • an implementation method for controlling the amount of output image data which may include the following steps:
  • the two control strategies in this embodiment may be enabled separately, or may be used in combination at the same time, to achieve the effect of adaptively controlling the image output quality.
  • the present application also provides another embodiment, which can adjust the quantization step size of the image coding process in real time in combination with the change trend of network congestion,
  • the image coding quality can be changed smoothly according to the network congestion, and the image display experience can be guaranteed when the network congestion is adjusted.
  • the congestion of the network is continuously detected, and the number P of image frames to be sent in the cache module is used as a quantitative representation of the congestion of the network.
  • the P value increases, it indicates that the congestion situation is increasing, and when the P value decreases, it indicates that the network congestion situation is decreasing.
  • this embodiment does not directly use the increase or decrease of the P value to directly control the selection of the quantization step size, but uses the change trend of the quantized P value to increase the quantization step size. Control amount.
  • the present application also uses a schematic example to describe the calculation process of the quantization step size, that is, an implementation manner of the step "adjusting the initial quantization step size based on the network congestion change information to obtain the quantization step size at the current moment" with reference to FIGS. 3-7 .
  • the quantization step size of P c can be
  • the horizontal axis is time
  • the vertical axis is the congestion level P value
  • the slope of the vector V2 from point B to point C is greater than the slope of the vector V1 from point A to point B
  • the correction value ⁇ Q step will be a positive number greater than 1, that is, ⁇ Q step >1. Therefore, Q step-c will perform greater corrections on the basis of Q step-b , further reducing the quality of image compression and the amount of image data, and promoting faster recovery from network congestion.
  • the correction value ⁇ Q step will be a positive number greater than 0 and less than 1, that is, 1> ⁇ Q step >0. Therefore, Q step-c will still be corrected on the basis of Q step-b , but the magnitude of the correction value is slowed down, the quality of image compression is slowed down, and the amount of compressed image data is reduced, but it still promotes the recovery of network congestion.
  • the correction value ⁇ Q step will be a number less than 0 at this time, that is, ⁇ Q step ⁇ 0, so Q step-c will still be corrected on the basis of Q step-b , but the correction will be reverse correction, Avoid excessively increasing the quantization step size for image compression when the congestion level has started to ease, resulting in unnecessary image quality degradation.
  • the correction value ⁇ Q step will be a number less than 0 and greater than -1, that is, -1 ⁇ Q step ⁇ 0, so Q step-c will still be corrected on the basis of Q step-b , but the Correction slowly to avoid excessively increasing the quantization step size for image compression when congestion levels have indeed started to ease, resulting in unnecessary image quality degradation.
  • the slope of the vector V1 from point A to point B is negative
  • the slope of vector V2 from point B to point C is also negative
  • the slope of V2 is greater than V1, indicating that the degree of congestion has begun to accelerate decline.
  • the correction value ⁇ Q step will be a number less than -1, that is, ⁇ Q step ⁇ -1. Therefore, Q step-c will accelerate the correction and recovery on the basis of Q step-b , and speed up the recovery of the normal quantization step size when the congestion level has indeed begun to accelerate and alleviate, so as to avoid unnecessary image quality degradation.
  • this embodiment adjusts the change trend of the quantization step size for image coding in time in combination with the change trend of the current network congestion degree, so as to achieve a slow change of the quantization step size, make the image coding quality transition smoothly, and avoid the sudden change of image quality affecting users. experience.
  • FIG. 8 and FIG. 9 may include:
  • This embodiment includes a BMC chip and a memory chip mounted on the BMC chip, and the image data of the local server host is transmitted to the BMC chip and then written into the memory chip.
  • this application adds an image judgment module for implementation, and modifies the control module, so that the entire image output path can be optimized according to the characteristics of the server interface image.
  • the function of the image compression module is to read the original image data from the memory, use the standard image compression method to compress the original image data, and then write the compressed data into the image cache module.
  • the method of image compression is processed according to the standard image coding method.
  • the compression process generally includes image segmentation, DCT transformation, quantization and entropy coding.
  • the DCT transform and quantization process have an impact on the image quality and the compressed data size.
  • the purpose of quantization is to reduce the image coding length without reducing the visual effect. The larger the quantization step size, the more information is lost after compression, and the smaller the amount of data, but the more "rough" the image display after decoding and restoration.
  • the image cache module is composed of storage resources in the BMC, and caches the compressed image data frames.
  • the image cache module can accommodate a certain number of frames of compressed image data, and update the size and cache address of each frame of compressed images in real time.
  • the output interface reads the compressed image data frame by frame from the image buffer module and transmits it to the remote end through the network.
  • the image judging module is used for acquiring the image change characteristics of the local server interface and judging the characteristics of the local server interface image, and then transmitting the judgment result to the control module.
  • the control module controls the image compression module and the image cache module to perform corresponding processing according to the judgment result obtained by the image judgment module. Generally speaking, if there is no congestion at the image output end, the control module does not interfere with the image compression module and the image buffering module. If the image output terminal is congested, the control module starts to control the image compression module and the image buffer module to reduce the data amount of the image output terminal.
  • the BMC can control the compression quality of the image compression module and the frame dropping behavior of the image cache module to effectively reduce the amount of data at the image output end and relieve the congestion of image output breakpoints.
  • the control module controls the image compression module to use a larger quantization step size when performing quantization, thereby reducing the image quality of the image compression process and reducing the data amount of the compressed image.
  • the present application further adds optimization control to the quantization step size of image compression.
  • the control module controls the image cache module to discard the image cache frame.
  • this embodiment can effectively reduce the data volume requirement at the image output end of the BMC, and alleviate the situation of network congestion at the image output end.
  • different methods are used to reduce the amount of data at the image output end, and the quality of the remote image display is adaptively optimized. While alleviating the congestion at the layer output end, it provides a good remote interface experience and provides better Optimized remote visualization experience.
  • the change trend of the quantization step size for image coding is adjusted in time according to the change trend of the current network congestion degree, so as to achieve a slow change of the quantization step size, so that the image coding quality can be adjusted smoothly, so as to avoid the sudden change of image quality affecting the user experience.
  • the embodiment of the present application also provides a corresponding apparatus for the image output method, which further makes the method more practical.
  • the device can be described from the perspective of functional modules and the perspective of hardware.
  • the following describes the image output device provided by the embodiments of the present application, and the image output device described below and the image output method described above may refer to each other correspondingly.
  • FIG. 10 is a structural diagram of an image output device provided by an embodiment of the present application in a specific implementation manner, and the device may include:
  • the image change information acquisition module 901 is configured to acquire the image continuous change feature of the display interface of the local server.
  • the image output control information generation module 902 is configured to generate image output control information according to the continuous change feature of the image and the corresponding image change threshold.
  • the image output control module 903 is configured to control the output image data amount according to the image output control information and the network congestion information.
  • the image output control information generation module 902 may include an image flatness enabling sub-module, and the sub-module may be used for:
  • the image flatness enabling sub-module can be further used for:
  • the above-mentioned image output control information generation module 902 may include a similarity enabling sub-module, and the sub-module may be used for:
  • the above-mentioned similarity enabling sub-module can be further used for:
  • the above-mentioned image output control module 903 may also be a module that calculates the quantization step size update value and/or generates the image cache frame discarding instruction according to the image output control information and the network congestion information. .
  • the above-mentioned image output control module 903 may further include, for example, a quantization step size calculation sub-module, which may be used for:
  • the network congestion change information is determined according to the numerical change relationship of the number of image frames to be sent at the current moment, the first moment and the second moment; the initial quantization step size is adjusted based on the network congestion change information to obtain the quantization step size at the current moment.
  • the above-mentioned apparatus may further trigger a module, and the trigger module is configured to trigger the image change information acquisition module 901 to start working after receiving the output control instruction.
  • the embodiments of the present application can effectively solve the current situation of poor output image quality caused by directly discarding image frames when the network is congested, can ensure the output image quality when adjusting the network congestion, and provide users with a better remote visualization experience.
  • a computer device is provided, and the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 11 .
  • the computer equipment includes a processor, memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system and computer-readable instructions.
  • the internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer readable instructions when executed by a processor, implement an image output method.
  • the display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen
  • the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.
  • the embodiments of the present application can effectively solve the current situation of poor output image quality caused by directly discarding image frames when the network is congested, can ensure the output image quality when adjusting the network congestion, and provide users with a better remote visualization experience.
  • a computer device includes a memory and one or more processors, wherein computer readable instructions are stored in the memory, and when executed by the processors, the computer readable instructions cause the one or more processors to perform the above method.
  • One or more non-volatile storage media storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the above method.
  • the embodiments of the present application can effectively solve the current situation of poor output image quality caused by directly discarding image frames when the network is congested, can ensure the output image quality when adjusting the network congestion, and provide users with a better remote visualization experience.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Discrete Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开了一种图像输出方法、装置及计算机可读存储介质。其中，方法包括获取本地服务器的显示界面的图像连续变化特征；根据图像连续变化特征和预设图像变化阈值生成图像输出控制信息；根据图像输出控制信息和网络拥塞信息控制输出图像数据量。

Description

图像输出方法、装置、计算机设备及计算机可读存储介质
相关申请的交叉引用
[根据细则91更正 12.01.2022]　
本申请要求在2021年01月07日提交中国专利局，申请号为202110019483.X，申请名称为“图像输出方法、装置及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别是涉及一种图像输出方法、装置、计算机设备及计算机可读存储介质。
背景技术
BMC(Baseboard Manager Controller，基板管理控制器)在平台管理中表示的是一系列的监视和控制功能，操作的对象是系统硬件，比如通过监视系统的温度，电压，风扇、电源等等，并做相应的调节工作，以保证系统处于健康的状态。其主要用于采集单个服务器上各种信息，同时通过两种方法提供给上层运维网管软件，第一种为BMC会提供各种各样的接口供上层网管查询，如web界面、命令行等人机接口；第二种为主动上报，当检测到有故障产生时，BMC可以通过网络等手段上报给上层网管软件的服务端，以便运维人员及时识别处理故障。
其中，BMC提供的web图形界面是一个很直观便利的交互接口形式，BMC支持KVM(Keyboard Video Mouse，键盘、视频和鼠标)over IP服务，通过KVM服务构建本地服务器和远程管理者之间的可视化界面。BMC芯片借助网络接口来传输可视化服务所需要的键盘、鼠标和图像数据。可以理解的是，若传输的图像数据较大，网络环境较复杂，直接将图像数据通过网络传输时易产生拥塞，影响服务器界面显示延时，进而影响远程可视化体验。所以会在BMC中会先对图像的帧数据进行压缩和缓存，再将压缩后的图像数据通过网络传输到远程。
相关技术在图像输出过程中，当图像输出端出现拥塞时，会采用一定的规则控制图像缓存的写入和读取方式，在输出端因阻塞而来不及处理图像缓存数据时，直接将未传送的缓存数据丢掉若干帧，以免阻塞状态向图像输入端传递，影响整个图像输出路径。
但是这种未考虑图像数据的特征在网络输出端出现拥塞时直接丢弃部分缓存帧的方 法，尽管处理过程简单，但对于网络出现较严重拥塞，且服务器界面图像变化较快时，发明人意识到，现有的图像输出方法丢弃的缓存图像帧会包含较多的界面变化信息，丢弃这些图像帧后会使得呈现可视化界面出现突变的现象，用户体验较差，无法满足用户到高质量输出图像的现实需求
发明内容
为解决上述技术问题，本申请实施例提供以下技术方案：
本申请实施例一方面提供了一种图像输出方法，包括：
获取本地服务器的显示界面的图像连续变化特征；
根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息；和
根据图像输出控制信息和网络拥塞信息控制输出图像数据量。
在其中一个实施例中，根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息包括：
在当前帧图像内变化平缓程度大于预设图像变化阈值时，生成图像压缩质量调节指令；和
在当前帧图像内变化平缓程度不大于预设图像变化阈值时，生成图像质量保持不变指令。
在其中一个实施例中，根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息，包括：
利用二维傅里叶变换或DCT变换计算当前帧图像的高频分量和低频分量；
在高频分量占高低频分量总数的比例值大于预设比例阈值时，确定当前帧图像内变化平缓程度大于预设图像变化阈值；和
在高频分量占高低频分量总数的比例值不大于预设比例阈值时，确定当前帧图像内变化平缓程度不大于预设图像变化阈值。
在其中一个实施例中，根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息包括：
在相邻图像之间的相似度大于预设图像相似度阈值时，生成显示界面不变的指令；和
在相邻图像之间的相似度不大于预设图像相似度阈值时，生成显示界面变化的指令。
在其中一个实施例中，根据图像连续变化特征和相应图像变化阈值生成图像输出控 制信息包括：
计算当前帧图像与上一帧图像之间的互信息；
在互信息大于预设图像相似度阈值时，确定相邻图像之间的相似度大于预设图像相似度阈值；和
在互信息不大于预设图像相似度阈值时，确定相邻图像之间的相似度不大于预设图像相似度阈值。
在其中一个实施例中，根据图像输出控制信息和网络拥塞信息控制输出图像数据量包括：
根据图像输出控制信息和网络拥塞信息计算量化步长更新值和/或生成图像缓存帧丢弃指令。
在其中一个实施例中，根据图像输出控制信息和网络拥塞信息计算量化步长更新值包括：
获取当前时刻、第一时刻和第二时刻的待发送图像帧数，第一时刻早于当前时刻，第二时刻早于第一时刻；
获取图像压缩过程中在第一时刻的初始量化步长；
根据当前时刻、第一时刻与第二时刻的待发送图像帧数的数值变化关系确定网络拥塞变化信息；和
基于网络拥塞变化信息调整初始量化步长，得到当前时刻的量化步长。
在其中一个实施例中，获取本地服务器的显示界面的图像连续变化特征之前，还包括：
在接收到输出控制指令时，执行获取本地服务器的显示界面的图像连续变化特征的步骤。
本申请实施例另一方面提供了一种图像输出装置，包括：
图像变化信息获取模块，用于获取本地服务器的显示界面的图像连续变化特征；
图像输出控制信息生成模块，用于根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息；和
图像输出控制模块，用于根据图像输出控制信息和网络拥塞信息控制输出图像数据量。
本申请实施例还提供了一种计算机设备，包括存储器及一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述一个或多个处理器执行时， 使得所述一个或多个处理器执行上述任一项图像输出方法的步骤。
本申请实施例最后还提供了一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行上述任一项图像输出方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚的说明本申请实施例或相关技术的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单的介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请根据一个或多个实施例中提供的一种图像输出方法的应用场景图；
图2为本申请根据一个或多个实施例提供的一种图像输出方法的流程示意图；
图3为本申请根据一个或多个实施例提供的示意性例子中的第一种网络拥塞程度变化曲线示意图；
图4为本申请根据一个或多个实施例提供的示意性例子中的第二种网络拥塞程度变化曲线示意图；
图5为本申请根据一个或多个实施例提供的示意性例子中的第三种网络拥塞程度变化曲线示意图；
图6为本申请根据一个或多个实施例提供的示意性例子中的第四种网络拥塞程度变化曲线示意图；
图7为本申请根据一个或多个实施例提供的示意性例子中的第五种网络拥塞程度变化曲线示意图；
图8为本申请根据一个或多个实施例提供的一个示例性应用场景的框架示意图；
图9为本申请根据一个或多个实施例提供的图像变化特征判断流程示意图；
图10为本申请根据一个或多个实施例提供的图像输出装置的一种具体实施方式结构图；
图11为本申请根据一个或多个实施例提供的计算机设备的内部结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面结合附图和具体实施方式对本申请作进一步的详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”“第四”等是用于区别不同的对象，而不是用于描述特定的顺序。此外术语“包括”和“具有”以及他们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可包括没有列出的步骤或单元。
在介绍了本申请实施例的技术方案后，下面详细的说明本申请的各种非限制性实施方式。
本申请提供的图像输出方法，可以应用于如图1所示的应用环境中。其中，终端102a与本地服务器104a通过网络进行通信。终端102a可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，服务器104a可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
具体地，终端102a获取本地服务器104a的显示界面的图像连续变化特征，根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息，和根据图像输出控制信息和网络拥塞信息控制输出图像数据量。
首先参见图2，图2为本申请实施例提供的一种图像输出方法的流程示意图，以该方法应用于图1中的终端为例进行说明，本申请实施例可包括以下内容：
S101：获取本地服务器的显示界面的图像连续变化特征。
本步骤中的图像连续变化特征可用于表示图像输出端输出的多帧相邻图像的变化情况，换句话来说，就是表示相邻多帧图像彼此之间的变化程度，变化程度即可包括色彩变化程度，也可包括内容变化程度，变化程度越小，证明相邻图像之间的差异性较小，即使丢弃一两帧图像或者是其中几帧图像质量不高，用户视觉也不容易察觉，可视化效果仍旧很好。变化程度越大，证明相邻图像之间的差异性越大，丢弃一两帧图像或者是其中几帧图像质量不高，用户很容易察觉，可视化效果不好。举例来说，上一帧图像为黑白图像，当前帧图像为油画图像，这就属于变化程度较大的两帧图像，上一帧图像为汽车图像，当前帧图像为材料金相图像，这也属于变化程度较大的两帧图像。上一帧图像为右视角下的彩色汽车图像，当前帧图像为左视角下的彩色汽车图像，这就属于变化程度不大的两帧图像。
S102：根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息。
可以理解的是，图像连续变化特征的表征参数不同，表征参数对应的图像变化阈值也就不同，所属领域技术人员可根据实际应用场景选择合适的图像连续变化特征和相对应的阈值，这均不影响本申请的实现。图像输出控制信息可用于表示要不要控制图像输出数据量以及当前显示图像的变化特征，如输出界面是基本不变还是正在变化，界面图像变化剧烈还是变化平缓。当然，图像输出控制信息还可根据实际应用场景设置包含其他参数。若有多个显示端，可同时在图像输出控制信息中携带相应显示端的标识信息。
S103：根据图像输出控制信息和网络拥塞信息控制输出图像数据量。
本步骤的网络拥塞信息用于表示图像输出端等待发送和/或即将要发送的图像数据量，控制输出图像数据量可包括通过丢弃图像数据帧减少输出图像数来降低图像数据总量输出，也可通过降低某些帧图像质量来降低图像数据总量输出，或者是同时通过丢弃图像数据帧和降低某些帧图像质量来减少图像数据总量输出，或者是采用其他方式来减少图像输出端的数据总量。
在本申请实施例提供的技术方案中，将本地服务器的显示界面图像的连续变化特征作为BMC调整图像编码过程的依据，在图像输出端出现拥塞时可通过减小图像编码帧数据量来缓解网络输出端的数据带宽需求，由于缓解网络输出端的数据带宽需求是基于输出端的显示图像的质量的，从而有效解决了网络堵塞时直接丢弃图像帧导致输出图像质量较差的现状，可在调节网络拥塞情况时保证输出图像的质量，为用户提供更优的远程可视化体验。
需要说明的是，本申请中各步骤之间没有严格的先后执行顺序，只要符合逻辑上的顺序，则这些步骤可以同时执行，也可按照某种预设顺序执行，图2只是一种示意方式，并不代表只能是这样的执行顺序。
可以理解的是，控制图像输出的情况不是一直都需要进行的，而整个过程即S101-S103是需要耗费系统资源，增加系统功耗，基于此，本申请在获取本地服务器的显示界面的图像连续变化特征之前，还可包括：
判断是否接收到输出控制指令；
若接收到输出控制指令，执行上述实施例中的S101-S103。
本步骤的输出控制指令就是用于指示要不要进行输出图像数据的控制，也即要不要执行S101-S103这个流程，所属领域技术人员可根据实际情况选择何时何种情况如网络堵塞时去触发执行S101-S103。举例来说，对于BMC来说，当BMC的图像输出模块未 出现拥塞情况时，可执行S101-S103，节省BMC芯片功耗，降低资源消耗。当图像输出模块出现拥塞情况再执行S101-S103的工作流程来减少图像输出端口的数据量。
在上述实施例中，对于如何执行步骤S102并不做限定，本实施例中给出图像输出控制信息生成方式，结合图9所示，可包括如下步骤：
本实施例的本地服务器界面图像的特征包括两种，一种为图像内部的亮度色度变化激烈程度，另一种为相邻图像之间的近似程度。
作为一种可选的实施方式，对于S101步骤中获取的本地服务器界面图像的特征为图像内部的亮度色度变化激烈程度的图像特征，图像内部的亮度色度变化激烈程度表示图像内变化平缓程度，例如一副纯色图像，其亮度色度变化特别平缓，而一副彩色油画，其亮度色度变化比较激烈。用压缩方法来处理纯色图像，可以使用更大压缩率的参数，而要处理彩色油画图像，必须使用低压缩率的参数，否则影响恢复后的图像质量。相应的，S102的实现过程可为：
判断当前帧图像内变化平缓程度是否大于预设图像变化阈值；
若当前帧图像内变化平缓程度大于预设图像变化阈值，生成图像压缩质量调节指令；
若当前帧图像内变化平缓程度不大于预设图像变化阈值，生成图像质量保持不变指令。
作为一种可选的实施方式，可采用高低频分量来定量评价图像内部的亮度色度变化激烈程度或者是说图像内变化平缓程度，相应的，步骤“判断当前帧图像内变化平缓程度是否大于预设图像变化阈值”的过程可包括：
利用二维傅里叶变换或DCT变换计算当前帧图像的高频分量和低频分量；判断高频分量占高低频分量总数的比例值是否大于预设比例阈值。
本实施例使用二维傅里叶变换或者DCT变换来计算图像的高低频分量，从而判断图像的变化平缓程度，如果图像变化剧烈，则图像的高频分量较大，如果图像变化平缓，则图像的低频分量较大。当相邻图像的变化激烈程度达到一定阈值时，则可控制图像缓存模块更改图像质量。
作为另一种可选的实施方式，与上述实施例并列，对于S101步骤中获取的本地服务器界面图像的特征为表征图像相似度的图像特征，相邻图像之间的近似程度表示下一个界面图像与上一个界面图像的近似程度。如果界面静止不变，则相邻图像完全相同，此时即使丢弃若干缓存图像帧，也不影响界面交互体验；而如果界面在剧烈变换，则相邻 图像的差异较大，此时如果还丢弃缓存帧，则界面会出现突变。相应的，S102的实现过程可为：
判断相邻图像之间的相似度是否大于预设图像相似度阈值；
若相邻图像之间的相似度大于预设图像相似度阈值，生成显示界面不变的指令；
若相邻图像之间的相似度不大于预设图像相似度阈值，生成显示界面变化的指令。
作为一种可选的实施方式，可采用互信息来评价相邻图像的相似度，相应的步骤“判断相邻图像之间的相似度是否大于预设图像相似度阈值”的过程可包括：
计算当前帧图像与上一帧图像之间的互信息；
判断互信息是否大于预设图像相似度阈值。
本实施例通过计算两幅图像的互信息来确认两幅图像的近似程度，互信息值越大，则表示两幅图像近似程度越大。若一段时间内图像的互信息达到一定阈值，则判定当前服务器显示界面基本保持不变，此时可使用丢弃缓存帧的方式来减少图像输出端的数据压力，缓解拥塞情况，同时不会明显影响远程的界面显示效果。当相邻图像的近似程度达到一定阈值时，则可控制图像缓存模块进行丢弃图像缓存帧。
在上述实施例中，对于如何执行步骤S103并不做限定，本实施例中给出一种控制输出图像数据量的实现方式，可包括如下步骤：
根据图像输出控制信息和网络拥塞信息计算量化步长更新值和/或生成图像缓存帧丢弃指令。
本实施例的这两种控制策略可以分别单独使能，也可以同时结合起来使用，达到自适应控制图像输出质量的效果。
为了进一步保证在缓解网络拥堵情况的同时不影响用户的可视化体验，基于上述实施例，本本申请还提供了另外一个实施例，其可结合网络拥塞的变化趋势实时调整图像编码过程的量化步长，使图像编码质量根据网络拥塞情况进行平稳的对应变化，可在调节网络拥塞情况时保证图像显示体验。可包括：
获取当前时刻、第一时刻和第二时刻的待发送图像帧数，第一时刻早于当前时刻，第二时刻早于第一时刻；
获取图像压缩过程中在第一时刻的初始量化步长；
根据当前时刻、第一时刻与第二时刻的待发送图像帧数的数值变化关系确定网络拥塞变化信息；
基于网络拥塞变化信息调整初始量化步长，得到当前时刻的量化步长。
本实施例会持续检测网络的拥塞情况，并使用缓存模块中待发送的图像帧数P作为网络拥塞情况的量化表示。当P值增大时，表示拥塞情况在增大，而P值减小时，表示网络拥塞情况在减小。此外为更平稳的控制图像质量，本实施例并不直接使用P值的增大或减小来直接控制量化步长的选取，而是使用量化后的P值的变化趋势来为量化步长增加控制量。本申请还结合图3-图7以一个示意性例子阐述量化步长的计算过程即步骤“基于网络拥塞变化信息调整初始量化步长，得到当前时刻的量化步长”的一种实现方式。
本实施例以均匀间隔检测三个点的拥塞值P a、P b和P c，并计算AB两点拥塞程度的插值Δba＝P b-P a，和BC两点拥塞程度的插值Δcb＝P c-P b。然后在选择量化步长时为量化步长Qstep增加一个修正值ΔQ step，ΔQ step的计算关系式可为：
Figure PCTCN2021134143-appb-000001
P c的量化步长可为
Figure PCTCN2021134143-appb-000002
其中，sign(Δcb)表示取Δcb的符号，当Δcb为正时，sign(Δcb)＝1；当Δcb为负时，sign(Δcb)＝-1。
举例来说，如果网络拥塞程度按图3的曲线变化，图中横轴为时间，纵轴为拥塞程度P值，B点到C点的向量V2斜率大于A点到B点的向量V1斜率，可以判断出拥塞程度正在加速变大。修正值ΔQ step会是一个大于1的正数，即ΔQ step>1。所以Q step-c会在Q step-b的基础上进行更大的修正，进一步降低图像压缩的质量和图像数据量，促进网络拥塞情况更快恢复。
如果网络拥塞程度按图4曲线变化，B点到C点的向量V2斜率大于A点到B点的向量V1斜率，可以判断出拥塞程度仍在变大，但是有缓解的趋势。根据计算修正值ΔQ step会是一个大于0小于1的正数，即1>ΔQ step>0。所以Q step-c会在Q step-b的基础上仍然进行修正，但是修正值幅度放缓，图像压缩的质量下降程度放缓，压缩图像数据量降低，但仍促进网络拥塞情况恢复。
进一步地，如果网络拥塞程度开始按图5曲线变化，A点到B点的向量V1斜率为正值，但B点到C点的向量V2斜率已经开始变为负值，表示拥塞程度已经开始下降，图像传输通道拥塞有所缓解。根据计算，此时修正值ΔQ step会是一个小于0的数，即 ΔQ step<0，所以Q step-c会在Q step-b的基础上仍然进行修正，但是进行的修正是反向修正，在拥塞程度已经开始缓解的情况下避免图像压缩的量化步长过度增大，导致不必要的图像质量下降。
如果网络拥塞程度开始按图6曲线变化，A点到B点的向量V1斜率为负值，B点到C点的向量V2斜率也为负值，且V2斜率大于V1，表示拥塞程度已经开始加速下降。根据计算此时修正值ΔQ step会是一个小于0大于-1的数，即-1<ΔQ step<0，所以Q step-c会在Q step-b的基础上仍然进行修正，但是进行的是缓慢修正，在拥塞程度已经确实开始缓解的情况下避免图像压缩的量化步长过度增大，导致不必要的图像质量下降。
如果网络拥塞程度开始按图7曲线变化，A点到B点的向量V1斜率为负值，B点到C点的向量V2斜率也为负值，且V2斜率大于V1，表示拥塞程度已经开始加速下降。根据计算此时修正值ΔQ step会是一个小于-1的数，即ΔQ step<-1。所以Q step-c会在Q step-b的基础加速修正恢复，在拥塞程度已经确实开始加速缓解的情况下加快恢复正常的量化步长，避免导致不必要的图像质量下降。
由上可知，本实施例结合当前网络拥塞程度的变化趋势及时调整对图像编码的量化步长的变化趋势，以达到量化步长缓慢变化，使图像编码质量平稳过渡，以避免图像质量突变影响用户体验。
为了使所属领域技术人员更加清楚明白本申请的技术方案，本申请提供了一个示意性例子来阐述本申请的技术方案，请参阅图8及图9所示，可包括：
本实施例包括BMC芯片和BMC芯片挂载的内存芯片，本地服务器主机的图像数据传输到BMC芯片然后写入内存芯片。在BMC芯片内除通用的图像压缩模块、图像缓存模块、输出接口外，本申请为了实现还增加一个图像判断模块，并修改控制模块，使整个图像输出路径能根据服务器界面图像的特征进行优化。
一般来说，图像压缩模块的功能是从内存中读取原始的图像数据，使用标准的图像压缩方法对原始图像数据进行压缩，得到压缩后的数据再写入图像缓存模块。图像压缩的方法按照标准的图像编码方式进行处理。以JPEG为例，压缩的流程一般包括图像分割、DCT变换、量化和熵编码过程。其中DCT变换和量化过程对图像质量和压缩后数据大小有影响。以量化过程为例量化的目的在不降低视觉效果的前提下减少图像编码长度，量化步长越大，压缩后丢失的信息越多，数据量越小，但解码恢复后图像显示越“粗糙”；量化步长越小，丢失的信息越少，图像编码后数据量越大，但解码恢复后图像 显示越“精细”。图像缓存模块由BMC内存储资源构成，对压缩后的图像数据帧进行缓存。图像缓存模块可容纳一定数量帧的压缩图像数据，并实时更新每帧压缩图像的大小和缓存地址。输出接口从图像缓存模块中逐帧读取压缩图像数据通过网络传输到远端。图像判断模块用于获取本地服务器界面的图像变化特征并对本地服务器界面图像的特征进行判断，然后并将判断结果传递给控制模块。图像判断模块关注的特征主要有两种：图像内部的亮度色度变化激烈程度，以及相邻图像之间的近似程度。控制模块根据图像判断模块得到的判断结果来控制图像压缩模块和图像缓存模块进行相应的处理。一般来说，如果图像输出端没有拥塞情况，控制模块不干预图像压缩模块和图像缓存模块。如果图像输出端出现拥塞情况，则控制模块开始控制图像压缩模块和图像缓存模块来减小图像输出端的数据量。
BMC可控制图像压缩模块的压缩质量，以及图像缓存模块的丢帧行为，来有效减少图像输出端的数据量，缓解图像输出断点的拥塞情况。具体来说，当图像内变化平缓时，控制模块控制图像压缩模块在进行量化时使用更大的量化步长，从而降低图像压缩处理的图像质量，减少压缩后图像的数据量。为避免因BMC芯片网络接口突发的拥塞导致量化步长突然变化，进而使使图像质量出现跳变，本申请对图像压缩的量化步长进一步增加了优化控制。当相邻图像的近似程度达到一定阈值时，控制模块控制图像缓存模块进行丢弃图像缓存帧。
由上可知，本实施例可有效减少BMC图像输出端的数据量需求，缓解图像输出端出现网络拥塞的情况。根据本地服务器界面的图像特征，针对性采用不同的方法来减少图像输出端的数据量，自适应优化远程图像显示质量，在缓解图层输出端拥塞情况的同时，提供良好的远程界面体验，提供更优化的远程可视化体验。进一步的，结合当前网络拥塞程度的变化趋势，来及时调整对图像编码的量化步长的变化趋势，以达到量化步长缓慢变化，使图像编码质量平稳调节，以避免图像质量突变影响用户体验。
本申请实施例还针对图像输出方法提供了相应的装置，进一步使得所述方法更具有实用性。其中，装置可从功能模块的角度和硬件的角度分别说明。下面对本申请实施例提供的图像输出装置进行介绍，下文描述的图像输出装置与上文描述的图像输出方法可相互对应参照。
基于功能模块的角度，参见图10，图10为本申请实施例提供的图像输出装置在一种具体实施方式下的结构图，该装置可包括：
图像变化信息获取模块901，用于获取本地服务器的显示界面的图像连续变化特征。
图像输出控制信息生成模块902，用于根据图像连续变化特征和相应图像变化阈值生成图像输出控制信息。
图像输出控制模块903，用于根据图像输出控制信息和网络拥塞信息控制输出图像数据量。
可选的，在本实施例的一些实施方式中，上述图像输出控制信息生成模块902可以包括图像平缓程度使能子模块，该子模块可用于：
判断当前帧图像内变化平缓程度是否大于预设图像变化阈值；若当前帧图像内变化平缓程度大于预设图像变化阈值，生成图像压缩质量调节指令；若当前帧图像内变化平缓程度不大于预设图像变化阈值，生成图像质量保持不变指令。
作为本实施例的一种可选的实施方式，图像平缓程度使能子模块进一步可用于：
利用二维傅里叶变换或DCT变换计算当前帧图像的高频分量和低频分量；判断高频分量占高低频分量总数的比例值是否大于预设比例阈值。
在本实施例的另一些实施方式中，上述图像输出控制信息生成模块902可以包括相似度使能子模块，该子模块可用于：
判断相邻图像之间的相似度是否大于预设图像相似度阈值；若相邻图像之间的相似度大于预设图像相似度阈值，生成显示界面不变的指令；若相邻图像之间的相似度不大于预设图像相似度阈值，生成显示界面变化的指令。
作为本实施例的一种可选的实施方式，上述相似度使能子模块可进一步用于：
计算当前帧图像与上一帧图像之间的互信息；判断互信息是否大于预设图像相似度阈值。
可选的，在本实施例的其他一些实施方式中，上述图像输出控制模块903还可为根据图像输出控制信息和网络拥塞信息计算量化步长更新值和/或生成图像缓存帧丢弃指令的模块。
作为本实施例的一种可选的实施方式，上述图像输出控制模块903例如还可包括量化步长计算子模块，该子模块可用于：
获取当前时刻、第一时刻和第二时刻的待发送图像帧数，第一时刻早于当前时刻，第二时刻早于第一时刻；获取图像压缩过程中在第一时刻的初始量化步长；根据当前时刻、第一时刻与第二时刻的待发送图像帧数的数值变化关系确定网络拥塞变化信息；基于网络拥塞变化信息调整初始量化步长，得到当前时刻的量化步长。
可选的，在本实施例的其他一些实施方式中，上述装置例如还可触发模块，所述触 发模块用于在接收到输出控制指令后，触发图像变化信息获取模块901开始工作。
本申请实施例所述图像输出装置的各功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可以参照上述方法实施例的相关描述，此处不再赘述。
由上可知，本申请实施例可有效解决网络堵塞时直接丢弃图像帧导致输出图像质量较差的现状，在调节网络拥塞情况时可保证输出图像质量，为用户提供更优的远程可视化体验。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图11所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机可读指令。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种图像输出方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本申请实施例所述图像输出装置的各功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可以参照上述方法实施例的相关描述，此处不再赘述。
由上可知，本申请实施例可有效解决网络堵塞时直接丢弃图像帧导致输出图像质量较差的现状，在调节网络拥塞情况时可保证输出图像质量，为用户提供更优的远程可视化体验。
一种计算机设备，包括存储器和一个或多个处理器，存储器中储存有计算机可读指令，计算机可读指令被处理器执行时，使得一个或多个处理器执行上述方法。
一个或多个存储有计算机可读指令的非易失性存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述方法。
由上可知，本申请实施例可有效解决网络堵塞时直接丢弃图像帧导致输出图像质量较差的现状，在调节网络拥塞情况时可保证输出图像质量，为用户提供更优的远程可视化体验。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通 过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (11)

  1. 一种图像输出方法，其特征在于，包括：
    获取本地服务器的显示界面的图像连续变化特征；
    根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息；和
    根据所述图像输出控制信息和网络拥塞信息控制输出图像数据量。
  2. 根据权利要求1所述的图像输出方法，其特征在于，所述根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息包括：
    在当前帧图像内变化平缓程度大于预设图像变化阈值时，生成图像压缩质量调节指令；和
    在当前帧图像内变化平缓程度不大于预设图像变化阈值时，生成图像质量保持不变指令。
  3. 根据权利要求2所述的图像输出方法，其特征在于，所述根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息，还包括：
    利用二维傅里叶变换或DCT变换计算所述当前帧图像的高频分量和低频分量；
    在所述高频分量占高低频分量总数的比例值大于预设比例阈值时，确定所述当前帧图像内变化平缓程度大于所述预设图像变化阈值；和
    在所述高频分量占高低频分量总数的比例值不大于预设比例阈值时，确定所述当前帧图像内变化平缓程度不大于所述预设图像变化阈值。
  4. 根据权利要求1所述的图像输出方法，其特征在于，所述根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息包括：
    在相邻图像之间的相似度大于预设图像相似度阈值时，生成显示界面不变的指令；和
    在相邻图像之间的相似度不大于预设图像相似度阈值时，生成显示界面变化的指令。
  5. 根据权利要求4所述的图像输出方法，其特征在于，所述根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息，还包括：
    计算当前帧图像与上一帧图像之间的互信息；
    在所述互信息大于所述预设图像相似度阈值时，确定所述相邻图像之间的相似度大于预设图像相似度阈值；和
    在所述互信息不大于所述预设图像相似度阈值时，确定所述相邻图像之间的相似度不大于预设图像相似度阈值。
  6. 根据权利要求1至5任意一项所述的图像输出方法，其特征在于，所述根据所述图像输出控制信息和网络拥塞信息控制输出图像数据量包括：
    根据所述图像输出控制信息和所述网络拥塞信息计算量化步长更新值和/或生成图像缓存帧丢弃指令。
  7. 根据权利要求6所述的图像输出方法，其特征在于，所述根据所述图像输出控制信息和网络拥塞信息计算量化步长更新值包括：
    获取当前时刻、第一时刻和第二时刻的待发送图像帧数，所述第一时刻早于所述当前时刻，所述第二时刻早于所述第一时刻；
    获取图像压缩过程中在所述第一时刻的初始量化步长；
    根据所述当前时刻、所述第一时刻与所述第二时刻的待发送图像帧数的数值变化关系确定网络拥塞变化信息；和
    基于所述网络拥塞变化信息调整所述初始量化步长，得到所述当前时刻的量化步长。
  8. 根据权利要求6所述的图像输出方法，其特征在于，所述获取本地服务器的显示界面的图像连续变化特征之前，还包括：
    在接收到所述输出控制指令时，执行所述获取本地服务器的显示界面的图像连续变化特征的步骤。
  9. 一种图像输出装置，其特征在于，包括：
    图像变化信息获取模块，用于获取本地服务器的显示界面的图像连续变化特征；
    图像输出控制信息生成模块，用于根据所述图像连续变化特征和相应图像变化阈值生成图像输出控制信息；和
    图像输出控制模块，用于根据所述图像输出控制信息和网络拥塞信息控制输出图像数据量。
  10. 一种计算机设备，包括存储器及一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1-8任意一项所述的方法的步骤。。
  11. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1-8任意一项所述的方法的步骤。
PCT/CN2021/134143 2021-01-07 2021-11-29 图像输出方法、装置、计算机设备及计算机可读存储介质 WO2022148180A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/259,864 US20240320950A1 (en) 2021-01-07 2021-11-29 Image Output Method and Apparatus, Computer Device and Computer-Readable Storage Medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110019483.XA CN112804527B (zh) 2021-01-07 2021-01-07 图像输出方法、装置及计算机可读存储介质
CN202110019483.X 2021-01-07

Publications (1)

Publication Number Publication Date
WO2022148180A1 true WO2022148180A1 (zh) 2022-07-14

Family

ID=75808935

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/134143 WO2022148180A1 (zh) 2021-01-07 2021-11-29 图像输出方法、装置、计算机设备及计算机可读存储介质

Country Status (3)

Country Link
US (1) US20240320950A1 (zh)
CN (1) CN112804527B (zh)
WO (1) WO2022148180A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112804527B (zh) * 2021-01-07 2023-01-24 苏州浪潮智能科技有限公司 图像输出方法、装置及计算机可读存储介质
CN116208788B (zh) * 2023-05-04 2023-07-21 海马云（天津）信息技术有限公司 提供网络应用服务的方法及装置、服务器设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101853504A (zh) * 2010-05-07 2010-10-06 厦门大学 基于视觉特性与结构相似度的图像质量评测方法
CN102208106A (zh) * 2010-03-31 2011-10-05 富士通株式会社 图像匹配装置和图像匹配方法
US20160196478A1 (en) * 2013-09-03 2016-07-07 Samsung Electronics Co., Ltd. Image processing method and device
CN106375755A (zh) * 2016-08-31 2017-02-01 福建省天奕网络科技有限公司 一种视频数据卡顿检测方法及系统
CN110782442A (zh) * 2019-10-23 2020-02-11 国网陕西省电力公司宝鸡供电公司 一种基于多域耦合的图像人工模糊检测方法
CN111970518A (zh) * 2020-08-14 2020-11-20 山东云海国创云计算装备产业创新中心有限公司 一种图像丢帧处理方法、系统、设备及计算机存储介质
CN112804527A (zh) * 2021-01-07 2021-05-14 苏州浪潮智能科技有限公司 图像输出方法、装置及计算机可读存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1468001A (zh) * 2002-06-27 2004-01-14 上海汉唐科技有限公司 基于因特网的媒体流自适应传输方法
CN110418140A (zh) * 2019-07-26 2019-11-05 华北电力大学 视频的优化传输方法及系统
CN111212025B (zh) * 2019-11-20 2022-02-08 腾讯科技（深圳）有限公司 网络自适应视频流的传输方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102208106A (zh) * 2010-03-31 2011-10-05 富士通株式会社 图像匹配装置和图像匹配方法
CN101853504A (zh) * 2010-05-07 2010-10-06 厦门大学 基于视觉特性与结构相似度的图像质量评测方法
US20160196478A1 (en) * 2013-09-03 2016-07-07 Samsung Electronics Co., Ltd. Image processing method and device
CN106375755A (zh) * 2016-08-31 2017-02-01 福建省天奕网络科技有限公司 一种视频数据卡顿检测方法及系统
CN110782442A (zh) * 2019-10-23 2020-02-11 国网陕西省电力公司宝鸡供电公司 一种基于多域耦合的图像人工模糊检测方法
CN111970518A (zh) * 2020-08-14 2020-11-20 山东云海国创云计算装备产业创新中心有限公司 一种图像丢帧处理方法、系统、设备及计算机存储介质
CN112804527A (zh) * 2021-01-07 2021-05-14 苏州浪潮智能科技有限公司 图像输出方法、装置及计算机可读存储介质

Also Published As

Publication number Publication date
CN112804527A (zh) 2021-05-14
US20240320950A1 (en) 2024-09-26
CN112804527B (zh) 2023-01-24

Similar Documents

Publication Publication Date Title
US12108065B2 (en) Encoding method and apparatus for screen sharing, storage medium, and electronic device
WO2022148180A1 (zh) 图像输出方法、装置、计算机设备及计算机可读存储介质
US9712854B2 (en) Cost-aware cloud-based content delivery
WO2018132987A1 (zh) 一种亮度调节方法及终端
WO2017084255A1 (zh) 实时视频增强方法、终端和非易失性计算机可读存储介质
Yan et al. RnB: Rate and brightness adaptation for rate-distortion-energy tradeoff in HTTP adaptive streaming over mobile devices
EP2913767A1 (en) Page processing method, apparatus and system
US12022096B2 (en) Human visual system adaptive video coding
US8726054B2 (en) Cloud-based energy-saving service system and method
KR101805550B1 (ko) 프리젠테이션 가상화를 위한 화면 부호화 방법 및 서버
WO2023035939A1 (zh) 背光调节方法、装置、设备及存储介质
TWI586175B (zh) 視訊會議頻寬管理方法及系統
TW201347541A (zh) 視訊編碼系統、方法以及電腦可讀取之媒體
CN112752107A (zh) 网页图片预处理方法、系统、存储介质及计算机设备
WO2014067494A1 (zh) 处理图像的方法和设备
US9805662B2 (en) Content adaptive backlight power saving technology
CN112350998A (zh) 一种基于边缘计算的视频流传输方法
Bartolini et al. HVS-DBS: Human visual system-aware dynamic luminance backlight scaling for video streaming applications
Xiao et al. Reducing display power consumption for real-time video calls on mobile devices
CN115002462A (zh) 一种编码方法、装置、设备及存储介质
CN110351014B (zh) 数据处理方法、装置、计算机可读存储介质和计算机设备
Liu et al. Content-adaptive display power saving for internet video applications on mobile devices
CN113518188B (zh) 一种基于摄像系统nfc动态拼接方法以及装置
US10135896B1 (en) Systems and methods providing metadata for media streaming
CN111010525B (zh) 动态调整图像清晰度的方法及其图像处理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21917221

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18259864

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21917221

Country of ref document: EP

Kind code of ref document: A1