WO2023040825A1 - 媒体信息的传输方法、计算设备及存储介质 - Google Patents

媒体信息的传输方法、计算设备及存储介质 Download PDF

Info

Publication number
WO2023040825A1
WO2023040825A1 PCT/CN2022/118422 CN2022118422W WO2023040825A1 WO 2023040825 A1 WO2023040825 A1 WO 2023040825A1 CN 2022118422 W CN2022118422 W CN 2022118422W WO 2023040825 A1 WO2023040825 A1 WO 2023040825A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
updated
processing method
processing
threshold
Prior art date
Application number
PCT/CN2022/118422
Other languages
English (en)
French (fr)
Inventor
吴栋磊
程艳
闵洪波
郑文丹
李江卫
Original Assignee
阿里巴巴（中国）有限公司
阿里云计算有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202111101871.9A external-priority patent/CN113973224B/zh
Application filed by 阿里巴巴（中国）有限公司, 阿里云计算有限公司 filed Critical 阿里巴巴（中国）有限公司
Publication of WO2023040825A1 publication Critical patent/WO2023040825A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/458—Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations
    • H04N21/4586—Content update operation triggered locally, e.g. by comparing the version of software modules in a DVB carousel to the version stored locally

Definitions

  • the present application relates to the field of computer technology, and in particular to a method for transmitting media information, a computing device and a storage medium.
  • Various aspects of the present application provide a media information transmission method, computing device, and storage medium, so as to be able to use different data streams to update media information in different scenarios to meet update requirements, while maintaining user experience .
  • An embodiment of the present application provides a method for transmitting media information, including: acquiring an image to be updated; determining an image processing method according to the image to be updated, and the processing method makes the processed image to be updated meet the transmission conditions and/or The display scene of the image; according to the determined processing method, the image to be updated is processed and transmitted, so that the receiving end receives the processed image and displays it.
  • the embodiment of the present application also provides a method for transmitting media information, including: receiving the processed image to be updated, and obtaining the image to be updated according to the processed image; updating the image to be updated on the display screen; determining the image to be updated The time difference between the update time of the image to be updated on the display screen and the generation time of the image to be updated; when the time difference is greater than the time threshold and/or the bandwidth occupied by the processed image to be updated is greater than the bandwidth threshold, send The information is fed back to the sending end, so that after receiving the feedback information, the sending end determines that the processing mode of the image to be updated is the processing mode of the video stream according to the feedback information.
  • the embodiment of the present application also provides a computing device, including: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to: obtain an image to be updated; according to the Describe the image to be updated, determine the processing method of the image, the processing method makes the processed image to be updated meet the transmission conditions and/or the display scene of the image; according to the determined processing method, the image to be updated is processed and transmitted, so that the receiving The terminal receives the processed image and displays it.
  • a computing device including: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to: obtain an image to be updated; according to the Describe the image to be updated, determine the processing method of the image, the processing method makes the processed image to be updated meet the transmission conditions and/or the display scene of the image; according to the determined processing method, the image to be updated is processed and transmitted, so that the receiving The terminal receives the processed image and displays it.
  • the embodiment of the present application also provides a computing device, including: a memory, a processor, and a communication component; the memory is used to store a computer program; the communication component is used to receive the processed image to be updated; the processor , executing the computer program to: obtain the image to be updated according to the processed image; update the image to be updated on the display screen; determine the update time of the image to be updated on the display screen and the The time difference between the generation times of the images to be updated; the communication component is configured to send feedback information to the sending end when the time difference is greater than the time threshold and/or the bandwidth occupied by the processed image to be updated is greater than the bandwidth threshold, to After receiving the feedback information, the sending end determines that the processing mode of the image to be updated is the processing mode of the video stream according to the feedback information.
  • a computing device including: a memory, a processor, and a communication component; the memory is used to store a computer program; the communication component is used to receive the processed image to be updated; the processor , executing the computer program
  • the embodiment of the present application also provides a computer-readable storage medium storing a computer program.
  • the computer program is executed by one or more processors, the one or more processors are caused to implement the steps in the above method.
  • the image to be updated is obtained; according to the image to be updated, the processing method of the image is determined, and the processing method makes the processed image to be updated meet the transmission conditions and/or the display scene of the image; according to the determined processing method, the processing method of the image is determined Update the image for processing and transmission, so that the receiving end receives the processed image and displays it.
  • the image to be updated is processed and transmitted according to the determined processing method, and the processing method makes the processed image to be updated meet the transmission condition and/or the display scene of the image.
  • the processed image to be updated can satisfy the display scene of the image and at the same time meet the transmission conditions, so that different processing methods can be determined according to the requirements of different display scenes to process the image to be updated and generate corresponding data Stream and transmit to meet different update requirements and maintain a good user experience.
  • FIG. 1 is a schematic flowchart of a method for transmitting media information according to an exemplary embodiment of the present application
  • FIG. 2 is a schematic diagram of the transmission of media information in an exemplary embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a media information transmission system according to an exemplary embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a method for transmitting media information according to an exemplary embodiment of the present application
  • FIG. 5 is a schematic structural diagram of a device for transmitting media information provided by an exemplary embodiment of the present application
  • FIG. 6 is a schematic structural diagram of a device for transmitting media information provided by an exemplary embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a computing device provided by an exemplary embodiment of the present application.
  • Fig. 8 is a schematic structural diagram of a computing device provided by an exemplary embodiment of the present application.
  • the embodiments of the present application provide a media information transmission method, a computing device, and a storage medium, which may be able to.
  • Fig. 1 is a schematic flowchart of a method for transmitting media information according to an exemplary embodiment of the present application.
  • the method 100 provided by the embodiment of the present application is executed by a computing device, such as a server.
  • the method 100 includes the following steps:
  • the processing method makes the processed image to be updated satisfy the transmission condition and/or the display scene of the image.
  • the server may be in various forms, such as a cloud server or a common physical machine server.
  • the image to be updated refers to an image used for updating on the display screen.
  • the display screen may belong to the user terminal or a display screen of the client. For example, a computer screen.
  • a user logs in to a video server (the video server may be a cloud server) through a terminal device, such as a computer, installed with a web browser client to watch online movies.
  • the user clicks on the desired movie through the browser client to watch it.
  • the picture of the movie will be continuously updated, and at this time, the update of the image frame or video frame will be involved.
  • the video server needs to regularly or continuously send the image to be updated to the computer for updating the image of the movie.
  • the video server starts to send the images to be updated of the movie, which may be sent regularly, and one transmission may be multiple frames of images. In this way, the video server can spontaneously acquire the corresponding image to be updated and send it.
  • the video server needs to render and generate the image to be updated before sending the image to be updated.
  • Rendering and generation can be performed by existing methods, or by the following methods.
  • acquiring the image to be updated includes: receiving an image instruction, and acquiring the image to be updated from the instruction.
  • the video application on the video server receives a request to play movie A sent by the user through the computer of the browser client, and then the application needs to send a rendering request to the local image engine of the server, so that the image engine can render according to the It is requested to render the picture of the movie A, so as to generate the corresponding video picture.
  • the graphics engine sends an instruction or command to the local driver of the server, so that the driver converts the instruction or command into a preset instruction or command, such as the command of the qxl image command stream, which has a corresponding generated image, that is, to be updated image.
  • the drive can then send the command to a command queue to wait for processing. Therefore, the corresponding image to be updated can be obtained from the command.
  • an image update scenario of text editing such as text editing in a web page.
  • an image update scenario of normal text web page browsing for example, an image update scene of web page browsing on a browser.
  • an image update scene in a small area for example, an image update scene in a small area (less than a certain area threshold) in a web page.
  • an image update scene in a large area for example, an image update scene in a large area (greater than a certain area threshold) in a web page.
  • the image update scene in the window animation for example, the image update scene in the window animation in the web page, and the image update scene in the remote desktop operation, etc.
  • the image update scenes corresponding to various update scenes or display scenes will not be listed one by one. All these scenarios can be realized through the embodiments of the present application.
  • the terminal device installed with the client can send an update request to the corresponding server (similar to the above), after the corresponding server receives the request, according to the method described above Render the corresponding screen, and then convert and issue instructions, etc., so I won’t go into details.
  • the server it can have different forms of servers according to different scenarios, such as video servers, text webpage servers, etc.
  • These servers can all be servers used to create dynamic interactive webpages and establish powerful webpage application programs, for example, all can be servers deployed with dynamic server pages.
  • the rendering of the image to be updated it may be implemented by other servers, or by the server of the execution device, such as a video server. If implemented by other servers, the above instructions or commands may be sent by other servers to the server of the execution device, such as a video server.
  • the processing method makes the processed image to be updated satisfy the transmission condition and/or the display scene of the image.
  • the transmission condition may refer to a transmission bandwidth condition, such as a bandwidth threshold and the like.
  • the display scene can be as described above, so I won’t repeat it here.
  • the processing manner may include a processing manner of a graphics command stream (ie, an image command stream) and a processing manner of a video stream.
  • the graphics command stream may be processed by compressing the acquired image to be updated to generate a compressed graphics command stream.
  • the compression method may include: lz4 (a lossless compression algorithm), gzip (GNUzip, a file compression), jpeg (Joint Photographic Experts Group, a compression standard), and its transmission method may include: quic (Quick UDP Internet Connection, a low-latency Internet transport layer protocol based on UDP), http (Hyper Text Transfer Protocol, hypertext transfer protocol), tcp (Transmission Control Protocol, transmission control protocol), udp (User Datagram Protocol, user datagram agreement), etc.
  • the video stream may be processed by encoding the acquired image to be updated to generate an encoded video stream.
  • the encoding methods can include: mjpeg (Motion Joint Photographic Experts Group, a video encoding format), h265 (a video encoding format), vp8 (a video compression format), vp9 (a video compression format), av1 (a video coding standard), etc.
  • the transmission method can also be as described above.
  • the graphics command flow is a graphics command flow for spice (Simple Protocol for Independent Computing Environment, Simple Protocol for Independent Computing Environment).
  • determining the image processing method according to the image to be updated includes: determining the processing method according to the size of the image to be updated, the update time of the image to be updated in the display scene, and/or the bandwidth data occupied by the processed image to be updated
  • the mode is the processing mode of the image command stream or the processing mode of the video stream.
  • the processing method can be determined by determining whether the display scene and/or bandwidth conditions are met according to the size of the image to be updated, the update time of the image to be updated in the presentation scene, and/or the bandwidth data occupied by the processed image to be updated.
  • the processing method of the video stream can be used; when the update time is too long (it can be compared with the corresponding threshold to determine whether it is too long), the video stream can be used.
  • the bandwidth data is too large (it can be compared with the corresponding threshold to determine whether it is too large), the processing method of the video stream can be used.
  • determining the processing method of the image according to the image to be updated includes: determining the area of the image to be updated; determining whether the area is greater than the area threshold; If it is greater than the area threshold, it is determined that the processing method is the processing method of the video stream.
  • the video server 201 receives the command of the qxl image command stream described above, which may be sent by the other server mentioned above.
  • the video server 201 receives the command through the mode converter 202 therein, and the image to be updated can be obtained from the command.
  • the command may also carry information about the image, such as the size of the image, the length and width of the image, and then the mode converter 202 determines the area according to the size. When the area is greater than the area threshold, it is determined that the processing mode is the processing mode of the video stream. If the current processing method is already the video stream processing method, there is no need to change it.
  • the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode. I won't go into too much detail.
  • the area threshold can be determined according to different types of compression ratios used and empirical values of network transmission, or can be determined through data delay.
  • determine the image processing method including: determine the area of the image to be updated, and determine the proportion of the area in the display screen; determine whether the proportion is greater than the proportion threshold, and when it is less than or equal to the proportion threshold, then determine the processing
  • the processing method is the processing method of the graphics command stream; when it is greater than the ratio threshold, it is determined that the processing method is the processing method of the video stream.
  • the video server 201 receives the command through the mode converter 202 therein, and the image to be updated can be obtained from the command.
  • the command may also carry the information of the image, such as the size of the image, the length and width of the image, and then the mode converter 202 determines the area according to the size.
  • the instruction may also carry the size of the display screen, and the mode converter 202 may determine the corresponding area according to the size.
  • the mode converter 202 determines the proportion of the image to be updated on the display screen according to the area of the image to be updated and the area of the display screen.
  • the processing method is the processing method of the video stream. If the current processing method is already the video stream processing method, there is no need to change it. If the current processing method is the processing method of the graphics command stream, you need to switch the processing method, and switch the processing method of the graphics command stream to the processing method of the video stream. . Otherwise, it is determined that the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode. I won't go into too much detail.
  • the above ratio may also be carried in the instruction or command.
  • the size of the display screen it can be based on the model of the smart terminal that the smart terminal with the client can carry when sending a request to the corresponding server, and the server can determine the model from the preset information of multiple devices The corresponding display screen size.
  • the threshold of the above ratio can be dynamically adjusted, and can also be obtained through data experiments.
  • the currently used update scene or display scene can be predicted based on the above-mentioned area and ratio, and the processing method can be switched in advance based on the predicted update scene or display scene.
  • determine the processing method of the image including: within a preset time, determine the area corresponding to the multiple images to be updated, and determine the sum of the areas of the multiple images to be updated; determine whether the sum of the areas is greater than When the threshold of the sum of areas is less than or equal to the threshold of the sum of areas, it is determined that the processing method is the processing method of the graphics command stream; when it is greater than the threshold of the sum of the areas, the processing method of the video stream is determined.
  • the preset time may refer to a preset cycle time, such as a refresh cycle.
  • the video server 201 may receive a plurality of the above-mentioned commands through the mode converter 202 therein, and the corresponding to-be-updated command may be obtained from each command. image.
  • the command may also carry the information of the image, such as the size of the image, the length and width of the image, and then the mode converter 202 determines the area of each image according to the size. Then, the mode converter 202 can determine or count the sum of areas of multiple images to be updated within a preset refresh period.
  • the processing mode is the processing mode of the video stream. If the current processing method is already the video stream processing method, there is no need to change it. If the current processing method is the processing method of the graphics command stream, you need to switch the processing method, and switch the processing method of the graphics command stream to the processing method of the video stream. . Otherwise, it is determined that the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode. I won't go into too much detail.
  • the commands or instructions of the above-mentioned qxl image command stream (that is, the graphics command stream), it can carry at least one image to be updated, that is to say, at least one image to be updated is generated during rendering.
  • at least one image to be updated corresponding to each image to be updated has the same size.
  • the receiving end refers to a device for receiving an image update, such as a computer, and may also be a client in the device.
  • the video server processes the corresponding image to be updated according to the processing method, and then transmits it to the user's computer, so that the computer receives the processed image to be updated, and obtains The image is to be updated, and the image is updated so that the user can see the update of the movie.
  • the image to be updated is processed and transmitted, including: when the determined processing method is the processing method of the graphics command stream, the image to be updated is compressed according to the compression method of the graphics command stream, and the compressed The image to be updated is transmitted; when the determined processing method is the processing method of the video stream, the image to be updated is encoded according to the encoding method of the video stream, and the encoded image to be updated is transmitted.
  • the mode converter 202 in the video server 201 processes the image to be updated after determining the processing mode.
  • the mode converter 202 performs image compression on the image to be updated, such as image compression through the jpeg standard, and generates the corresponding graphics command stream 203, that is, step 211 is executed: generating graphics command flow.
  • the graphics command stream 203 is sent to the user's computer through the above-mentioned transmission protocol, that is, step 213 is executed: sending the graphics command stream for updating and displaying.
  • the mode converter 202 performs video coding on the image to be updated, such as encoding the image to be updated by mjpeg video coding, and generates the corresponding video stream 204, that is, step 212 is executed: generate video stream. Then, the video stream 204 is sent to the user's computer through the above-mentioned transmission protocol, that is, step 214 is executed: sending the video stream for updating display.
  • the default processing method may be the processing method of the graphics command stream
  • the method 100 further includes: determining the sum of sizes of multiple compressed images to be updated within a preset time; when the sum of sizes is less than or equal to the threshold of the sum of sizes, the current compressed image to be updated will be transmitted; when the sum of sizes is greater than the threshold of the sum of sizes, the transmission of the currently compressed image to be updated will be stopped, and the processing of the current image to be updated will be determined
  • the method is the processing method of the video stream; the image to be updated is acquired, the image to be updated is encoded according to the encoding method of the video stream, and the encoded image to be updated is transmitted.
  • the threshold of the sum of sizes may be a bandwidth threshold of the network.
  • the processing method of the graphics command stream after determining the processing method of the graphics command stream, perform jpeg standard compression on the image to be updated. If the size of the compressed image to be updated exceeds the bandwidth threshold of the network, the compressed image to be updated in the processing mode of the graphics command stream will be discarded, and the processing mode will be changed to the processing mode of the video stream to process the image to be updated.
  • video encoding That is to say, the image to be updated can be obtained again through the command of the above-mentioned qxl image command flow, and then video encoding is performed for transmission.
  • the processed image to be updated is transmitted in the form of a protocol packet.
  • the browser client of the computer When the user receives the image to be updated after the above processing through the computer, if the image is processed according to the processing method of the graphics command stream, the browser client of the computer will decompress it through the corresponding processing method, such as the jpeg compression standard , and updated to the browser client for display, so that the video or image of the movie is updated, that is, the movie is continuously played. If the image is processed according to the processing method of the video stream, the browser client of the computer decodes it through the corresponding processing method, such as mjpeg video encoding, and updates it to the browser client for display, so that the video of the movie or The image is updated, i.e. the movie is played continuously. At this time, the browser client can determine whether to switch the processing mode according to the update delay and/or the received protocol packet.
  • the corresponding processing method such as the jpeg compression standard
  • the method 100 further includes: receiving feedback information sent by the receiving end, and determining the processing method as the video stream processing method according to the feedback information, wherein the receiving end determines the processing method according to the update delay and/or the processed image to be updated.
  • the update delay refers to the difference between the time when the image to be updated is generated and the time when the image is updated and displayed at the receiving end.
  • the bandwidth data occupied by the processed image to be updated may refer to the bandwidth data occupied by the corresponding protocol package.
  • the browser client after the browser client receives the protocol data, in addition to the image update described above, it can also perform delay statistics based on the generation time of the image to be updated and the final display time on the client In addition, the size of the received protocol packet is counted, and the bandwidth data size of the network is counted.
  • a QOE Quality of Experience, experience Quality
  • the feedback is passed to the video server, and the feedback information can be expressed as a processing method for switching to the video stream.
  • the mode converter 202 in the video server 201 executes step 215 : receiving feedback information, and then switching the processing mode to the video stream processing mode according to the feedback information.
  • the current processing method already belongs to the processing method of the graphics command stream, if the feedback information is received, it will switch to the processing method of the video stream, and if the current processing method belongs to the video stream, the time of receiving the feedback information will be updated, that is Updated the time to switch to the video stream processing method (even if it is currently the video stream processing method, it is still the latest time to update the processing method).
  • the clients involved in the embodiments of the present application are all directed at clients deployed with dynamic server pages.
  • the generation time of the image to be updated may be sent together with the protocol packet, which may be carried by the command of the above-mentioned qxl image command stream.
  • the bandwidth is monitored on the server and the client, the actual traffic corresponding to the above processing method can be controlled below a certain bandwidth threshold.
  • the client monitors the delay to form a closed loop of monitoring to ensure user experience.
  • the method 100 further includes: if no feedback information is received within the preset time, if the current processing mode is the video stream processing mode, switching the processing mode to the graphics command stream processing mode.
  • the current processing method is the video stream processing method
  • the current processing method is the video stream processing method
  • the preset refresh time exits the video stream processing method and switch to the graphics
  • the processing mode of the command stream is to process the image to be updated.
  • the image quality of the updated image can be improved, the bandwidth can be reduced, and the duration of video stream processing can be reduced through the above transmission link, thereby saving the cost of video encoding.
  • Fig. 3 is a schematic structural diagram of a media information transmission system provided by an exemplary embodiment of the present application. As shown in FIG. 3 , the system 300 may include: a first device 301 and a second device 302 .
  • the first device 301 refers to a device that can provide computing and processing services in a network virtual environment, and may refer to a device that uses a network to transmit media information.
  • the first device 301 may be any device capable of providing computing services, responding to service requests, and transmitting media information, such as cloud servers, cloud hosts, virtual centers, conventional servers, and so on.
  • the composition of the first device 301 mainly includes a processor, a hard disk, a memory, a system bus, etc., and is similar to a general computer architecture.
  • the second device 302 may be a device with a certain computing capability, and may realize a function of sending data to the first device 301 , and may also receive data sent by the first device 301 .
  • the basic structure of the second device 302 may include: at least one processor. The number of processors may depend on the configuration and type of device having a certain computing power. Devices with certain computing capabilities may also include memory, which may be volatile, such as RAM, or non-volatile, such as read-only memory (Read-Only Memory, ROM), flash memory, etc., or may also be Both types are included.
  • An operating system Operating System, OS
  • one or more application programs, and program data may also be stored in the memory.
  • devices with certain computing capabilities also include some basic configurations, such as network card chips, IO buses, display components, and some peripheral devices.
  • some peripheral devices may include, for example, a keyboard, a stylus, and the like.
  • Other peripheral devices are well known in the art and will not be described here.
  • the second device 302 may be a smart terminal, for example, a mobile phone, a desktop computer, a notebook, a tablet computer, and the like.
  • the first device 301 acquires the image to be updated; determines the processing method of the image according to the image to be updated, and the processing method makes the processed image to be updated meet the transmission conditions and the display scene of the image; according to the determined processing method, treat Update the image for processing and transmission, so that the receiving end receives the processed image and displays it.
  • the first device 301 receives an image instruction, and acquires an image to be updated from the instruction.
  • the first device 301 determines the area of the image to be updated; determines whether the area is greater than the area threshold, and if it is less than or equal to the area threshold, then determine that the processing mode is the processing mode of the graphics command stream; when it is greater than the area threshold, then determine the processing mode Indicates how the video stream is processed.
  • the first device 301 determines the area of the image to be updated, and determines the proportion of the area in the display screen; determines whether the proportion is greater than the proportion threshold, and if it is less than or equal to the proportion threshold, then determine that the processing method is the graphics command stream The processing method; when it is greater than the ratio threshold, it is determined that the processing method is the processing method of the video stream.
  • the first device 301 determines the areas corresponding to the multiple images to be updated, and determines the sum of the areas of the multiple images to be updated; determines whether the sum of the areas is greater than the threshold of the sum of the areas, and when it is less than or equal to the threshold of the sum of areas, then determine that the processing mode is the processing mode of the graphics command stream; when it is greater than the threshold of the sum of areas, then determine that the processing mode is the processing mode of the video stream.
  • the first device 301 when the determined processing mode is the processing mode of the graphics command stream, compresses the image to be updated according to the compression mode of the graphics command stream, and transmits the compressed image to be updated; when the determined processing If the method is the processing method of the video stream, the image to be updated is encoded according to the encoding method of the video stream, and the encoded image to be updated is transmitted.
  • the first device 301 determines the sum of sizes of multiple compressed images to be updated within a preset time; when the sum of sizes is less than or equal to If the threshold of the sum of sizes is set, the current compressed image to be updated will be transmitted; when the sum of sizes is greater than the threshold of the sum of sizes, the transmission of the currently compressed image to be updated will be stopped, and the processing method of the current image to be updated will be determined as
  • the processing method of the video stream obtain the image to be updated, encode the image to be updated according to the encoding method of the video stream, and transmit the encoded image to be updated.
  • the first device 301 receives the feedback information sent by the second device 302, and determines the processing method as the video stream processing method according to the feedback information, wherein the second device 302 determines the processing method according to the received update delay and/or the image to be updated. Or the feedback information sent by the bandwidth data occupied by the processed image to be updated.
  • the first device 301 if the first device 301 does not receive the feedback information within the preset time, if the current processing mode is the video stream processing mode, it switches the processing mode to the graphics command stream processing mode.
  • the second device 302 receives the processed image to be updated, and obtains the image to be updated according to the processed image; updates the image to be updated on the display screen; determines the update time of the image to be updated on the display screen and The time difference between the generation times of the images to be updated (that is, the update delay); when the time difference is greater than the time threshold and/or the bandwidth occupied by the processed image to be updated is greater than the bandwidth threshold, send feedback information to the sending end, so that the first After receiving the feedback information, the device 301 determines, according to the feedback information, that the processing mode of the image to be updated is the processing mode of the video stream.
  • the user logs in to the first device 301, such as the video server (the The video server can be a cloud server) to watch online movies.
  • the user clicks on the desired movie through the browser client to watch it.
  • the picture of the movie will be continuously updated, and at this time, the update of the image frame or video frame will be involved.
  • the video server needs to regularly or continuously send the image to be updated to the computer for updating the image of the movie.
  • the video server starts to send the images to be updated of the movie, which may be sent regularly, and one transmission may be multiple frames of images. In this way, the video server can spontaneously acquire the corresponding image to be updated and send it.
  • the video application on the video server receives the user's request to play movie A sent by the browser client computer, and then the application needs to send a rendering request to the local image engine of the server, so that the image engine renders the image of movie A according to the rendering request , so as to generate the corresponding video picture.
  • the graphics engine sends an instruction or command to the local driver of the server, so that the driver converts the instruction or command into a preset instruction or command, such as the command of the qxl image command stream, which has a corresponding generated image, that is, to be updated image.
  • the drive can then send the command to a command queue to wait for processing. Therefore, the corresponding image to be updated can be obtained from the command.
  • the command may be sent by the aforementioned other server.
  • the video server receives the command through the mode converter 202 therein, and the image to be updated can be obtained from the command.
  • the command may also carry information about the image, such as the size of the image, the length and width of the image, and then the mode converter 202 determines the area according to the size. When the area is greater than the area threshold, it is determined that the processing mode is the processing mode of the video stream. If the current processing method is already the video stream processing method, there is no need to change it.
  • the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode.
  • the mode converter 202 may also determine the proportion of the image to be updated on the display screen according to the area of the image to be updated and the area of the display screen.
  • the ratio is greater than the ratio threshold, it is determined that the processing method is the processing method of the video stream. If the current processing method is already the video stream processing method, there is no need to change it. If the current processing method is the processing method of the graphics command stream, you need to switch the processing method, and switch the processing method of the graphics command stream to the processing method of the video stream. . Otherwise, it is determined that the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode. I won't go into too much detail.
  • the mode converter 202 can also determine or count the sum of areas of multiple images to be updated within a preset refresh period. When the sum of the areas is greater than the threshold of the sum of areas, it is determined that the processing mode is the processing mode of the video stream. If the current processing method is already the video stream processing method, there is no need to change it. If the current processing method is the processing method of the graphics command stream, you need to switch the processing method, and switch the processing method of the graphics command stream to the processing method of the video stream. . Otherwise, it is determined that the processing mode is the processing mode of the graphics command stream. According to the above, if it is necessary to switch the processing mode, switch it, and if it is not necessary to maintain the processing mode. I won't go into too much detail.
  • the mode converter 202 in the video server processes the image to be updated after determining the processing mode.
  • the mode converter 202 performs image compression on the image to be updated, such as image compression through the jpeg standard, and generates the corresponding graphics command stream 203, that is, step 211 is executed: generating graphics command flow.
  • the graphics command stream 203 is sent to the user's computer through the above-mentioned transmission protocol, that is, step 213 is executed: sending the graphics command stream for updating and displaying.
  • the mode converter 202 performs video coding on the image to be updated, such as encoding the image to be updated by mjpeg video coding, and generates the corresponding video stream 204, that is, step 212 is executed: generate video stream.
  • the video stream 204 is sent to the user's computer through the above-mentioned transmission protocol, that is, step 214 is executed: sending the video stream for updating display.
  • the above sending manner may be sending by using a protocol packet.
  • the processing method of the graphics command stream After determining the processing method of the graphics command stream, perform jpeg standard compression on the image to be updated. After the jpeg standard compression is completed, count the size of all images to be updated after being compressed by the processing method of the graphics command stream in the above-mentioned one refresh cycle , if the bandwidth threshold of the network is exceeded, the compressed image to be updated in the processing mode of the graphics command stream is abandoned, and the processing mode is switched to the processing mode of the video stream, and the video encoding of the image to be updated is performed. That is, the image to be updated can be obtained again through the command of the above qxl image command stream and then video encoded for transmission.
  • the browser client After the browser client receives the protocol data, in addition to the image update described above, it can also perform delay statistics based on the generation time of the image to be updated and the final display time on the client. In addition, the received The size of the protocol packet is counted, and the bandwidth data size of the network is counted. When the statistical update delay and/or the statistical bandwidth data exceed the corresponding threshold, a QOE (Quality of Experience, quality of experience) feedback message is generated. Feedback Passed to the video server, the feedback information can be expressed as the processing method of switching to the video stream.
  • the mode converter 202 in the video server executes step 215: receiving feedback information, and then switching the processing mode to the video stream processing mode according to the feedback information.
  • the current processing method already belongs to the processing method of the graphics command stream, if the feedback information is received, it will switch to the processing method of the video stream, and if the current processing method belongs to the video stream, the time of receiving the feedback information will be updated, that is Updated the time to switch to the video stream processing method (even if it is currently the video stream processing method, it is still the latest time to update the processing method).
  • the current processing method is the processing method of the video stream
  • the preset refresh time exit the processing method of the video stream and switch to the processing method of the graphics command stream. Update image processing.
  • the first device 301 and the second device 302 are connected to a network. If the first device 301 and the second device 302 are connected by communication, the network standard of the mobile network can be 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE) , 4G+(LTE+), WiMax, 5G, etc.
  • GSM 2G
  • GPRS 2.5G
  • 3G WCDMA, TD-SCDMA, CDMA2000, UTMS
  • 4G LTE
  • 4G+(LTE+) WiMax
  • 5G etc.
  • an exemplary embodiment of the present application provides a method for transmitting media information.
  • the method provided in the embodiment of the present application is executed by a computing device, such as a computer.
  • the method 400 includes the following steps:
  • the sender can refer to the server mentioned above.
  • the execution subject may also be other devices, such as other interactive devices such as a server.
  • Fig. 5 is a schematic structural framework diagram of a device for transmitting media information provided by an exemplary embodiment of the present application.
  • the apparatus 500 can be applied to a server.
  • the device 500 includes: an acquisition module 501, a determination module 502, and a processing module 503; the functions of each module are described in detail below:
  • the acquiring module 501 is configured to acquire an image to be updated.
  • the determining module 502 is configured to determine a processing method of the image according to the image to be updated, and the processing method makes the processed image to be updated satisfy the transmission condition and/or the display scene of the image.
  • the processing module 503 is configured to process and transmit the image to be updated according to the determined processing method, so that the receiving end receives the processed image and displays it.
  • the determining module 502 is configured to determine that the processing mode is image command stream processing according to the size of the image to be updated, the update time of the image to be updated in the display scene, and/or the bandwidth data occupied by the processed image to be updated mode or how the video stream is processed.
  • the obtaining module 501 is configured to receive an image instruction, and obtain an image to be updated from the instruction.
  • the determining module 502 includes: a first determining unit, configured to determine the area of the image to be updated; determine whether the area is greater than the area threshold, and if it is less than or equal to the area threshold, determine that the processing mode is the processing mode of the graphics command stream; If it is greater than the area threshold, it is determined that the processing mode is the processing mode of the video stream.
  • the determining module 502 includes: a second determining unit, configured to determine the area of the image to be updated, and determine the proportion of the area in the display screen; determine whether the proportion is greater than the proportion threshold, and when it is less than or equal to the proportion threshold, determine the processing
  • the processing method is the processing method of the graphics command stream; when it is greater than the ratio threshold, it is determined that the processing method is the processing method of the video stream.
  • the determining module 502 includes: a third determining unit, configured to determine the areas corresponding to multiple images to be updated within a preset time, and determine the sum of the areas of the multiple images to be updated; determine whether the sum of the areas is greater than When the threshold of the sum of areas is less than or equal to the threshold of the sum of areas, it is determined that the processing method is the processing method of the graphics command stream; when it is greater than the threshold of the sum of the areas, the processing method of the video stream is determined.
  • a third determining unit configured to determine the areas corresponding to multiple images to be updated within a preset time, and determine the sum of the areas of the multiple images to be updated; determine whether the sum of the areas is greater than When the threshold of the sum of areas is less than or equal to the threshold of the sum of areas, it is determined that the processing method is the processing method of the graphics command stream; when it is greater than the threshold of the sum of the areas, the processing method of the video stream is determined.
  • the processing module 503 includes: a first processing unit, configured to compress the image to be updated according to the compression method of the graphics command stream when the determined processing method is the processing method of the graphics command stream, and compress the compressed image to be updated The image is transmitted; the second processing unit is configured to encode the image to be updated according to the encoding method of the video stream when the determined processing method is the processing method of the video stream, and transmit the encoded image to be updated.
  • the determination module 502 is also configured to determine the sum of the sizes of multiple compressed images to be updated within a preset time; the device 500 also includes : The transmission module is used to transmit the currently compressed image to be updated when the sum of sizes is less than or equal to the threshold of the sum of sizes; the determination module 502 is also used to transfer the image to be updated when the sum of sizes is greater than the threshold of the sum of sizes Stop transmitting the currently compressed image to be updated, and determine that the processing method of the current image to be updated is the processing method of the video stream; the acquisition module 501 is also used to obtain the image to be updated, and encode the image to be updated according to the encoding method of the video stream , and transmit the encoded image to be updated.
  • the determining module 502 is also configured to receive feedback information sent by the receiving end, and determine the processing method as the video stream processing method according to the feedback information, wherein the receiving end is based on the update delay and/or the processed image received by the image to be updated The feedback information sent by the bandwidth data occupied by the image to be updated.
  • the determination module 502 is further configured to switch the processing mode to a graphics command stream processing mode if the current processing mode is a video stream processing mode if no feedback information is received within a preset time.
  • Fig. 6 is a schematic structural framework diagram of a device for transmitting media information provided by an exemplary embodiment of the present application.
  • the apparatus 600 may be applied to a terminal device.
  • the device 600 includes: a receiving module 601, an updating module 602, a determining module 603, and a sending module 604; the functions of each module are described in detail below:
  • the receiving module 601 is configured to receive the processed image to be updated, and obtain the image to be updated according to the processed image.
  • the update module 602 is configured to update the image to be updated on the display screen.
  • the determination module 603 is configured to determine the time difference between the update time of the image to be updated on the presentation screen and the generation time of the image to be updated.
  • the sending module 604 is configured to send feedback information to the sending end when the time difference is greater than the time threshold and/or the bandwidth occupied by the processed image to be updated is greater than the bandwidth threshold, so that the sending end receives the feedback information and, according to the feedback information It is determined that the processing mode of the image to be updated is the processing mode of the video stream.
  • the structure of the apparatus 500 shown in FIG. 5 can be implemented as a computing device, such as a server.
  • the device 700 may include: a memory 701 and a processor 702;
  • the memory 701 is used for storing computer programs.
  • the processor 702 is configured to execute a computer program for: acquiring an image to be updated; determining a processing method of the image according to the image to be updated, and the processing method enables the processed image to be updated to meet transmission conditions and/or display of the image Scene: According to the determined processing method, the image to be updated is processed and transmitted, so that the receiving end receives the processed image and displays it.
  • the processor 702 is specifically configured to: determine the processing mode as image command stream according to the size of the image to be updated, the update time of the image to be updated in the display scene, and/or the bandwidth data occupied by the processed image to be updated or how the video stream is processed.
  • the processor 702 is specifically configured to: receive an image instruction, and obtain an image to be updated from the instruction.
  • the processor 702 is specifically configured to determine the area of the image to be updated; determine whether the area is greater than the area threshold; Then it is determined that the processing mode is the processing mode of the video stream.
  • the processor 702 is specifically configured to: determine the area of the image to be updated, and determine the proportion of the area in the display screen; determine whether the proportion is greater than the proportion threshold, and when it is less than or equal to the proportion threshold, determine that the processing method is graphics command flow The processing method; when it is greater than the ratio threshold, it is determined that the processing method is the processing method of the video stream.
  • the processor 702 is specifically configured to: within a preset time, determine the areas corresponding to the multiple images to be updated, and determine the sum of the areas of the multiple images to be updated; determine whether the sum of the areas is greater than the threshold of the sum of the areas , when it is less than or equal to the threshold of the sum of the areas, it is determined that the processing method is the processing method of the graphics command stream; when it is greater than the threshold of the sum of the areas, it is determined that the processing method is the processing method of the video stream.
  • the processor 702 is specifically configured to: when the determined processing mode is the processing mode of the graphics command stream, compress the image to be updated according to the compression mode of the graphics command stream, and transmit the compressed image to be updated; If the determined processing method is the processing method of the video stream, the image to be updated is encoded according to the encoding method of the video stream, and the encoded image to be updated is transmitted.
  • the processor 702 is also configured to: determine the sum of the sizes of the compressed images to be updated within a preset time; is less than or equal to the threshold of the sum of sizes, the currently compressed image to be updated will be transmitted; when the sum of sizes is greater than the threshold of the sum of sizes, the transmission of the currently compressed image to be updated will be stopped, and the current image to be updated will be determined
  • the processing method is the processing method of the video stream; acquire the image to be updated, encode the image to be updated according to the encoding method of the video stream, and transmit the encoded image to be updated.
  • the device 700 may also include a communication component, configured to: receive feedback information sent by the receiving end, and determine the processing mode as the video stream processing mode according to the feedback information, wherein the receiving end is based on the received image to be updated Feedback information for updating delay and/or bandwidth data transmission occupied by the processed image to be updated.
  • a communication component configured to: receive feedback information sent by the receiving end, and determine the processing mode as the video stream processing mode according to the feedback information, wherein the receiving end is based on the received image to be updated Feedback information for updating delay and/or bandwidth data transmission occupied by the processed image to be updated.
  • the processor 702 is further configured to: switch the processing mode to a graphics command stream processing mode if the current processing mode is a video stream processing mode if no feedback information is received within a preset time.
  • an embodiment of the present invention provides a computer storage medium.
  • a computer program is executed by one or more processors, one or more processors are caused to implement a method for transmitting media information in the method embodiment shown in FIG. 1-2. step.
  • the structure of the apparatus 600 shown in FIG. 6 can be implemented as a computing device, such as a computer.
  • the device 800 may include: a memory 801, a processor 802, and a communication component 803;
  • the memory 801 is used to store computer programs.
  • the communication component 803 is configured to receive the processed image to be updated.
  • the processor 802 is configured to execute a computer program to: obtain an image to be updated according to the processed image; update the image to be updated on the display screen; determine the update time of the image to be updated on the display screen and the image to be updated The time difference between the generation times of .
  • the communication component 803 is configured to send feedback information to the sender when the time difference is greater than the time threshold and/or the bandwidth occupied by the processed image to be updated is greater than the bandwidth threshold, so that the sender can determine the image to be updated according to the feedback information after receiving the feedback information.
  • the processing method of updating the image is the processing method of the video stream.
  • an embodiment of the present invention provides a computer storage medium.
  • the computer program is executed by one or more processors, one or more processors are caused to implement the steps of a method for transmitting media information in the method embodiment in FIG. 4 .
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative efforts.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable multimedia data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the The instruction means implements the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable multimedia data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device
  • the instructions provide steps for implementing the functions specified in the procedure or procedures of the flowchart and/or the block or blocks of the block diagram.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent storage in computer-readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash random access memory
  • Computer-readable media including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information.
  • Information may be computer readable instructions, data structures, modules of a program, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例提供一种媒体信息的传输方法、计算设备及存储介质。在本申请实施例中，获取待更新图像；根据待更新图像，确定图像的处理方式，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。其中，根据确定的处理方式，对待更新图像进行处理并传输，且处理方式使得处理后的待更新图像可满足传输条件以及图像的展示场景。从而使得处理后的待更新图像可以满足图像的展示场景的同时，还可以满足传输条件来满足不同的更新需求，维持用户良好的体验感。

Description

媒体信息的传输方法、计算设备及存储介质
本申请要求2021年09月18日递交的申请号为202111101871.9、发明名称为“媒体信息的传输方法、计算设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种媒体信息的传输方法、计算设备及存储介质。
背景技术
对于媒体信息的更新而言，如，图像更新，其可以具有不同的更新场景，如文本编辑中的图像更新、正常文本网页浏览中的图像更新、视频播放中的图像更新等。但是在不同场景下，其媒体信息更新的要求也是有差别的。所以在能够保证用户体验不下降的情况下，可以在不同场景下使用不同的数据流来进行媒体信息的更新，来满足不同的更新要求。而如何在不同场景下使用进行不同数据流是需要解决的问题。
发明内容
本申请的多个方面提供一种媒体信息的传输方法、计算设备及存储介质，用以能够满足在不同场景下使用不同的数据流来更新媒体信息来满足更新需求，同时还可以维持用户的体验。
本申请实施例提供一种媒体信息的传输方法，包括：获取待更新图像；根据所述待更新图像，确定图像的处理方式，所述处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
本申请实施例还提供一种媒体信息的传输方法，包括：接收处理后的待更新图像，并根据处理后的图像得到待更新图像；将所述待更新图像在展示屏幕上进行更新；确定所述待更新图像在展示屏幕上的更新时间与所述待更新图像的生成时间之间的时间差；当所述时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到所述反馈信息后，根据所述反馈信息确定待更新图像的处理方式为视频流的处理方式。
本申请实施例还提供一种计算设备，包括：存储器、处理器；所述存储器，用于存储计算机程序；所述处理器，执行所述计算机程序，以用于：获取待更新图像；根据所述待更新图像，确定图像的处理方式，所述处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
本申请实施例还提供一种计算设备，包括：存储器、处理器以及通信组件；所述存储器，用于存储计算机程序；所述通信组件，用于接收处理后的待更新图像；所述处理 器，执行所述计算机程序，以用于：根据处理后的图像得到待更新图像；将所述待更新图像在展示屏幕上进行更新；确定所述待更新图像在展示屏幕上的更新时间与所述待更新图像的生成时间之间的时间差；所述通信组件，用于当所述时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到所述反馈信息后，根据所述反馈信息确定待更新图像的处理方式为视频流的处理方式。
本申请实施例还提供一种存储有计算机程序的计算机可读存储介质，计算机程序被一个或多个处理器执行时，致使所述一个或多个处理器实现上述方法中的步骤。
在本申请实施例中，获取待更新图像；根据待更新图像，确定图像的处理方式，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
其中，根据确定的处理方式，对待更新图像进行处理并传输，且处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景。从而使得处理后的待更新图像可以满足图像的展示场景的同时，还可以满足传输条件，由此可以根据不同的展示场景的需求确定不同的处理方式，来对待更新图像进行处理，生成对应的数据流并进行传输，来满足不同的更新需求，维持用户良好的体验感。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1为本申请一示例性实施例的媒体信息的传输方法的流程示意图；
图2为本申请一示例性实施例的媒体信息的传输的示意图；
图3为本申请一示例性实施例的媒体信息的传输系统的结构示意图；
图4为本申请一示例性实施例的媒体信息的传输方法的流程示意图；
图5为本申请一示例性实施例提供的媒体信息的传输装置的结构示意图；
图6为本申请一示例性实施例提供的媒体信息的传输装置的结构示意图；
图7为本申请一示例性实施例提供的计算设备的结构示意图；
图8为本申请一示例性实施例提供的计算设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
根据前文所述可知，对于媒体信息的更新而言，如，图像更新以及视频更新等，其可以具有不同的更新场景，如文本编辑中的图像更新、正常文本网页浏览中的图像更新、 视频播放中的图像或视频更新等。但是在不同场景下，其媒体信息更新的要求也是有差别的。所以在能够保证用户体验不下降的情况下，可以在不同场景下使用不同的处理方式生成的数据流来进行媒体信息的更新，来满足不同的更新要求。而如何在不同场景下使用进行不同数据流是需要解决的问题。
基于此，本申请实施例提供了一种媒体信息的传输方法、计算设备及存储介质，可以能够。
下面结合方法实施例，针对媒体信息的传输媒体信息的传输过程进行详细说明。
图1为本申请一示例性实施例的一种媒体信息的传输方法的流程示意图。本申请实施例提供的该方法100由计算设备执行，如，服务器等。该方法100包括以下步骤：
101：获取待更新图像。
102：根据待更新图像，确定图像的处理方式。
其中，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景。
103：根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
需要说明的是，该服务器可以是多种多样的表现形态，如云服务器以及普通物理机服务器均可等。
以下针对上述步骤进行详细地阐述：
101：获取待更新图像。
其中，待更新图像是指用于在展示屏幕上进行更新的图像。该展示屏幕可以属于用户端或客户端的展示屏幕。如，电脑屏幕。
例如，用户通过终端设备，如电脑，安装的web网页浏览器客户端登录视频服务器(该视频服务器可以是云服务器)，进行线上电影的观看。用户通过该浏览器客户端点击想看的电影进行观看。在观看的过程中，电影的画面会持续更新，那么此时就会涉及到图像帧或视频帧的更新。则此时需要视频服务器定时或持续地向电脑发送待更新图像，用于更新电影的图像。则在用户选择了自己想要看的电影后，视频服务器开始发送该电影的待更新图像，可以是定时发送这些待更新图像，一次发送可以是多帧图像。由此视频服务器可以自发的获取到对应的待更新图像进行发送。
但是视频服务器在发送待更新图像前需要对待更新图像进行渲染和生成。可以通过现有的方式进行渲染和生成，也可以通过下述方式进行渲染和生成。
具体的，获取待更新图像，包括：接收图像指令，从指令中获取待更新图像。
例如，根据前文所述，视频服务器上的视频应用接收到用户通过浏览器客户端的电脑发送的播放电影A的请求，然后该应用需要向该服务器本地的图像引擎发送渲染请求，使得图像引擎根据渲染请求渲染该电影A的画面，从而生成对应视频画面。然后，图形引擎向服务器本地的驱动器发送指令或命令，以使驱动器将该指令或命令转换成预置指 令或命令，如qxl图像命令流的命令，该命令中具有对应的生成图像，即待更新图像。然后驱动器可以将该命令发送至命令队列中，以等待处理。由此可以从该命令中获取到对应的待更新图像。
需要说明的是，除了上述视频更新外，还可以有其它更新场景或者说展示场景，如文本编辑的图像更新场景，例如在网页中进行文本编辑等。如正常文本网页浏览的图像更新场景，例如在浏览器上进行网页浏览的图像更新场景。如小区域中的图像更新场景，例如在网页中小区域(小于一定区域面积阈值)的图像更新场景。如大区域中的图像更新场景，例如在网页中大区域(大于一定区域面积阈值)的图像更新场景。如窗口动画中的图像更新场景，例如在网页中窗口动画的图像更新场景、以及远程桌面操作中的图像更新场景等。各种不同更新场景或展示场景对应的图像更新场景，就不再一一列举了。这些场景均可以通过本申请实施例来实现。仅说明：当用户触发界面的更新的时候，可以通过安装有客户端的终端设备发送更新请求至对应的服务器(与前文所述相似)，对应的服务器接收到该请求后，根据前文所述的方式进行渲染对应的画面，然后进行指令的转换以及下发等，就不再赘述了。
此外对于服务器而言，其可以根据不同的场景具有不同形式的服务器，如视频服务器、文本网页服务器等均可。这些服务器均可以是用来创建动态交互式网页并建立强大的web网页应用程序的服务器，如均可以是部署有动态服务器页面的服务器。
另，对于待更新图像的渲染而言，其可以是其它服务器进行实现的，也可以是本执行设备的服务器实现的，如视频服务器。如果是其它服务器实现的，则可以由其它服务器将上述指令或命令发送至本执行设备的服务器，如视频服务器。
102：根据待更新图像，确定图像的处理方式。
其中，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景。其中，传输条件可以是指传输的带宽条件，如带宽阈值等。展示场景可以如前文所述，就不再赘述了。处理方式可以包括图形命令流(即图像命令流)的处理方式以及视频流的处理方式。图形命令流的处理方式可以是通过获取到的待更新图像进行压缩，生成压缩后的图形命令流。其中，压缩方式可以包括：lz4(一种无损压缩算法)、gzip(GNUzip，一种文件压缩)、jpeg(Joint Photographic Experts Group，一种压缩标准)，其传输方式可以包括：quic(Quick UDP Internet Connection，一种基于UDP的低时延的互联网传输层协议)、http(Hyper Text Transfer Protocol，超文本传输协议)、tcp(Transmission Control Protocol，传输控制协议)、udp(User Datagram Protocol，用户数据报协议)等。视频流的处理方式可以是通过获取到的待更新图像进行编码，生成编码后的视频流。其中编码方式可以包括：mjpeg(Motion Joint Photographic Experts Group，是一种视频编码格式)、h265(一种视频编码格式)、vp8(一种视频压缩格式)、vp9(一种视频压缩格式)、av1(一种视频编码标准)等。其传输方式也可以如前文所述。其中，图形命令流是针对 spice(Simple Protocol for Independent Computing Environment，独立计算环境简单协议)的图形命令流。
基于此，可以通过待更新图像确定上述处理方式：
具体的，根据待更新图像，确定图像的处理方式，包括：根据待更新图像的尺寸、待更新图像在展示场景中的更新时间和/或处理后的待更新图像所占用的带宽数据，确定处理方式为图像命令流的处理方式或视频流的处理方式。
可以根据待更新图像的尺寸、待更新图像在展示场景中的更新时间和/或处理后的待更新图像所占用的带宽数据确定是否满足展示场景和/或带宽条件，从而来确定处理方式。当尺寸过大(可以与对应的阈值相比较确定是否过大)则可以使用视频流的处理方式，当更新时间过长(可以与对应的阈值相比较确定是否过长)则可以使用视频流的处理方式，当带宽数据过大(可以与对应的阈值相比较确定是否过大)则可以使用视频流的处理方式。
具体的，根据待更新图像，确定图像的处理方式，包括：确定待更新图像的面积；确定面积是否大于面积阈值，当小于或等于面积阈值，则确定处理方式为图形命令流的处理方式；当大于面积阈值，则确定处理方式为视频流的处理方式。
例如，根据前文所述，如图2所示，视频服务器201在接收到前文所述的qxl图像命令流的命令，该命令可以是由前文所述其它服务器发送过来的。视频服务器201通过其中的模式转换器202接收到该命令，从该命令中可以获取到待更新图像。该命令中除了携带有该待更新图像，还可以携带有该图像的信息，如图像的尺寸，图像的长宽，然后模式转换器202根据该尺寸确定该面积。当该面积大于面积阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。就不再过多赘述了。
此外，面积阈值可以根据所使用的不同类型的压缩比和网络传输的经验值确定，也可以通过数据时延确定。
具体的，根据待更新图像，确定图像的处理方式，包括：确定待更新图像的面积，并确定面积在展示屏幕中的比例；确定比例是否大于比例阈值，当小于或等于比例阈值，则确定处理方式为图形命令流的处理方式；当大于比例阈值，则确定处理方式为视频流的处理方式。
例如，根据前文所述，如图2所示，视频服务器201通过其中的模式转换器202接收到该命令，从该命令中可以获取到待更新图像。该命令中除了携带有该待更新图像，还可以携带有该图像的信息，如图像的尺寸，图像的长宽，然后模式转换器202根据该 尺寸确定该面积。此外，该指令还可以携带有展示屏幕的尺寸，模式转换器202可以根据该尺寸确定对应的面积。由此，模式转换器202根据待更新图像的面积以及展示屏幕的面积，确定待更新图像在展示屏幕上的比例。当该比例大于比例阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。就不再过多赘述了。
还需要说明的是，上述比例也可以是指令或命令中携带的。对于展示屏幕的尺寸而言，其可以是根据具有客户端的智能终端在向对应的服务器发送请求时，可以携带有智能终端的型号，服务器可以根据该型号从预置的多个设备的信息中确定对应的展示屏幕尺寸。
上述比例的阈值可以动态调整，也可以通过数据试验得到。
由此，可以通过上述面积以及比例，对当前所使用的更新场景或展示场景，进行预判，基于预判的更新场景或展示场景，进行处理方式的提前切换。
具体的，根据待更新图像，确定图像的处理方式，包括：在预置时间内，确定多个待更新图像对应的面积，并确定多个待更新图像的面积之和；确定面积之和是否大于面积之和的阈值，当小于或等于面积之和的阈值，则确定处理方式为图形命令流的处理方式；当大于面积之和的阈值，则确定处理方式为视频流的处理方式。
其中，预置时间可以是指一个预置周期时间，如刷新周期。
例如，根据前文所述，如图2所示，在预置时间周期中，视频服务器201可以通过其中的模式转换器202接收到多个上述命令，从每个命令中可以获取到对应的待更新图像。该命令中除了携带有该待更新图像，还可以携带有该图像的信息，如图像的尺寸，图像的长宽，然后模式转换器202根据尺寸确定每张图像的面积。然后，模式转换器202可以确定或统计在预置刷新周期内，多个待更新图像的面积之和。当该面积之和大于面积之和阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。就不再过多赘述了。
需要说明的是，对于上述qxl图像命令流(即图形命令流)的命令或指令而言，其可以携带至少一张待更新图像，也就是说在渲染的时候生成了至少一张待更新图像。对于视频更新场景或其它场景而言，通常至少一张待更新图像中对应的每张待更新图像的尺寸都是一样的。
103：根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
其中，接收端是指接收用于更新图像的设备，如电脑，也可以是该设备中的客户端。
例如，根据前文所述，视频服务器在确定了处理方式后，根据处理方式对对应的待更新图像进行处理，然后传输至用户的电脑，以使电脑接收到该处理后的待更新图像后，得到待更新图像，并进行图像的更新，以使用户可以看到该电影的更新。
具体的，根据确定的处理方式，对待更新图像进行处理并传输，包括：当确定的处理方式为图形命令流的处理方式，则根据图形命令流的压缩方式，对待更新图像进行压缩，将压缩后的待更新图像进行传输；当确定的处理方式为视频流的处理方式，则根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
例如，根据前文所述，如图2所示，视频服务器201中的模式转换器202在确定了处理方式后，对待更新图像进行处理。当处理方式是图形命令流的处理方式，则模式转换器202，针对待更新图像进行图像压缩，如通过jpeg标准进行图像压缩，并生成对应的图形命令流203，即执行了步骤211：生成图形命令流。然后将图形命令流203通过上述传输协议发送至用户的电脑，即执行了步骤213：发送图形命令流，以进行更新展示。
当处理方式是视频流的处理方式，则模式转换器202，针对待更新图像进行视频编码，如通过mjpeg视频编码对待更新图像进行编码，并生成对应的视频流204，即执行了步骤212：生成视频流。然后将视频流204通过上述传输协议发送至用户的电脑，即执行了步骤214：发送视频流，以进行更新展示。
此外，根据前文可知，如果上述最终确定的处理方式还是图形命令流的处理方式，或者说一直都没有触发视频流的处理方式(此处可以默认处理方式为图形命令流的处理方式)，也可以通过下述方式继续确定是否要进行视频流的处理方式的切换。
具体的，在根据图形命令流的压缩方式，对待更新图像进行压缩后，该方法100还包括：在预置时间内，确定多个压缩后的待更新图像的大小之和；当大小之和小于或等于大小之和的阈值，则将当前压缩后的待更新图像进行传输；当大小之和大于大小之和的阈值，将停止传输当前压缩后的待更新图像，并确定当前待更新图像的处理方式为视频流的处理方式；获取待更新图像，根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
其中，大小之和的阈值可以是网络的带宽阈值。
例如，根据前文所述的方式，在确定是图形命令流的处理方式后，针对待更新图像进行jpeg标准压缩，当完成jpeg标准压缩后，统计在上述一个刷新周期内所有进行图形命令流的处理方式压缩后的待更新图像的大小，如果超出网络的带宽阈值，则放弃图形命令流的处理方式的压缩后的待更新图像，并将处理方式进行换到视频流的处理方式，进行待更新图像的视频编码。即可以重新通过上述qxl图像命令流的命令获取待更新图 像然后进行视频编码，以进行传输。
需要说明的是，在传输的过程中，处理后的待更新图像是以协议包的形式进行传输的。
此外，通过引入一个刷新周期，在该周期内通过所有的待更新图像来统计是否需要对当前的处理方式进行切换或确定，支持多块非连续的更新区域，对于其它方案所无法切换的场景，本申请实施例都可以切换，如对于较快速滚动的未连续命中的更新场景，以及对于非连续变化区域时等更新场景下等。
当用户通过电脑接收到上述处理后的待更新图像后，如果该图像是根据图形命令流的处理方式进行处理的，则电脑的浏览器客户端通过对应的处理方式，如jpeg压缩标准进行解压缩，并更新到浏览器客户端上进行展示，使得电影的视频或图像进行了更新，即持续性地播放了该电影。如果该图像是根据视频流的处理方式进行处理的，则电脑的浏览器客户端通过对应的处理方式，如mjpeg视频编码进行解码，并更新到浏览器客户端上进行展示，使得电影的视频或图像进行了更新，即持续性地播放了该电影。此时浏览器客户端可以针对更新时延和/或接收到的协议包来确定是否进行处理方式的切换的。
具体的，该方法100还包括：接收接收端发送的反馈信息，根据反馈信息确定处理方式为视频流的处理方式，其中接收端是根据接收到的待更新图像的更新时延和/或处理后的待更新图像所占用的带宽数据发送的反馈信息。
其中，更新时延是指待更新图像生成的时间和在接收到端进行更新显示的时间之差。
处理后的待更新图像所占用的带宽数据可以是指对应的协议包所占用的带宽数据。
例如，根据前文所述，浏览器客户端收到协议数据后，除了上文所述的图像更新外，还可以根据待更新图像的生成时间和最后在客户端上的显示时间进行时延的统计，此外再将收到的协议包的大小进行统计，统计网络的带宽数据大小，当统计后的更新时延和/或统计的带宽数据超出对应的阈值时，则生成QOE(Quality of Experience，体验质量)的反馈消息，反馈传递给视频服务器，该反馈信息可以表示为切换到视频流的处理方式。如图2所示，使得视频服务器201中的模式转换器202执行了步骤215：接收反馈信息，然后根据该反馈信息将处理方式切换到视频流的处理方式。即当前的处理方式已经属于图形命令流的处理方式时，如果收到反馈信息，则切换成视频流的处理方式，如果当前已经属于视频流的处理方式，则更新接收到反馈信息的时间，即更新了切换到视频流的处理方式的时间(即使当前已经是视频流的处理方式了，也依旧作为更新处理方式的最新时间)。
需要说明的是，为了与前文相对应，本申请实施例中涉及到的客户端均是针对部署有动态服务器页面的客户端。
此外，对于待更新图像的生成时间可以是随着协议包一同发送过来的，其可以是由上述qxl图像命令流的命令携带而来的。
由于在上述服务器和客户端进行了带宽的监测，可以使得将上述处理方式对应的实际流量控制在一定的带宽阈值之下。同时，通过客户端来监测时延，来行成监测的闭环，保障用户体验感。
如果未接收到反馈信息，则可以不进行更新。但如果当前处理方式一直是视频流的处理方式，则可以退出该方式。
具体的，该方法100还包括：在预置时间内，未接收到反馈信息，如果当前的处理方式为视频流的处理方式，则将处理方式切换为图形命令流的处理方式。
例如，根据前文所述，当当前的处理方式是视频流的处理方式的时候，且如果在一定时间或预置刷新时间内都未接收到反馈信息，则退出视频流的处理方式，切换到图形命令流的处理方式进行对待更新图像的处理。
根据前文可知，在保证了用户的体验感后，通过上述传输链路可以提高了更新图像的画质、降低带宽，并减少了视频流的处理方式的使用时长，从而节约了视频编码的成本。
图3为本申请一示例性实施例提供的一种媒体信息的传输系统的结构示意图。如图3所示，该系统300可以包括：第一设备301以及第二设备302。
其中，第一设备301是指可以在网络虚拟环境中提供计算处理服务的设备，可以是指利用网络进行媒体信息传输的设备。在物理实现上，第一设备301可以是任何能够提供计算服务，响应服务请求，并进行媒体信息传输的设备，例如可以是云服务器、云主机、虚拟中心、常规服务器等等。第一设备301的构成主要包括处理器、硬盘、内存、系统总线等，和通用的计算机架构类似。
第二设备302可以是有一定计算能力的设备，可以实现向第一设备301发送数据的功能，也可以接收第一设备301发送的数据。第二设备302的基本结构可以包括：至少一个处理器。处理器的数量可以取决于具有一定计算能力装置的配置和类型。具有一定计算能力装置也可以包括存储器，该存储器可以为易失性的，例如RAM，也可以为非易失性的，例如只读存储器(Read-Only Memory，ROM)、闪存等，或者也可以同时包括两种类型。存储器内通常存储有操作系统(Operating System，OS)、一个或多个应用程序，也可以存储有程序数据等。除了处理单元和存储器之外，具有一定计算能力装置还包括一些基本配置，例如网卡芯片、IO总线、显示组件以及一些外围设备等。可选地，一些外围设备可以包括，例如键盘、输入笔等。其它外围设备在本领域中是众所周知的，在此不做赘述。可选地，第二设备302可以为智能终端，例如，手机、台式电脑、笔记本、平板电脑等。
具体的，第一设备301，获取待更新图像；根据待更新图像，确定图像的处理方式，处理方式使得处理后的待更新图像可满足传输条件以及图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
具体的，第一设备301，接收图像指令，从指令中获取待更新图像。
具体的，第一设备301，确定待更新图像的面积；确定面积是否大于面积阈值，当小于或等于面积阈值，则确定处理方式为图形命令流的处理方式；当大于面积阈值，则确定处理方式为视频流的处理方式。
具体的，第一设备301，确定所述待更新图像的面积，并确定面积在展示屏幕中的比例；确定比例是否大于比例阈值，当小于或等于比例阈值，则确定处理方式为图形命令流的处理方式；当大于比例阈值，则确定处理方式为视频流的处理方式。
具体的，第一设备301，在预置时间内，确定多个待更新图像对应的面积，并确定多个待更新图像的面积之和；确定面积之和是否大于面积之和的阈值，当小于或等于面积之和的阈值，则确定处理方式为图形命令流的处理方式；当大于面积之和的阈值，则确定处理方式为视频流的处理方式。
具体的，第一设备301，当确定的处理方式为图形命令流的处理方式，则根据图形命令流的压缩方式，对待更新图像进行压缩，将压缩后的待更新图像进行传输；当确定的处理方式为视频流的处理方式，则根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
此外，在根据图形命令流的压缩方式，对待更新图像进行压缩后，第一设备301，在预置时间内，确定多个压缩后的待更新图像的大小之和；当大小之和小于或等于大小之和的阈值，则将当前压缩后的待更新图像进行传输；当大小之和大于大小之和的阈值，将停止传输当前压缩后的待更新图像，并确定当前待更新图像的处理方式为视频流的处理方式；获取待更新图像，根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
具体的，第一设备301，接收第二设备302发送的反馈信息，根据反馈信息确定处理方式为视频流的处理方式，其中第二设备302是根据接收到的待更新图像的更新时延和/或处理后的待更新图像所占用的带宽数据发送的反馈信息。
此外，第一设备301，在预置时间内，未接收到反馈信息，如果当前的处理方式为视频流的处理方式，则将处理方式切换为图形命令流的处理方式。
具体的，第二设备302，接收处理后的待更新图像，并根据处理后的图像得到待更新图像；将待更新图像在展示屏幕上进行更新；确定待更新图像在展示屏幕上的更新时间与待更新图像的生成时间之间的时间差(即更新时延)；当时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使第一设备301接收到反馈信息后，根据反馈信息确定待更新图像的处理方式为视频流的处理方式。
需要说明的是，在系统300中未能完全阐述的内容请参见前文方法100中的内容，其具体的实施方式也请参见前文所述的方法100的具体实施方式，此处就不再赘述。
在本申请实施例的视频播放中的图像更新场景中，如图3所示，用户通过第二设备302，如电脑，安装的web网页浏览器客户端登录第一设备301，如视频服务器(该视频服务器可以是云服务器)，进行线上电影的观看。用户通过该浏览器客户端点击想看的电影进行观看。在观看的过程中，电影的画面会持续更新，那么此时就会涉及到图像帧或视频帧的更新。则此时需要视频服务器定时或持续地向电脑发送待更新图像，用于更新电影的图像。则在用户选择了自己想要看的电影后，视频服务器开始发送该电影的待更新图像，可以是定时发送这些待更新图像，一次发送可以是多帧图像。由此视频服务器可以自发的获取到对应的待更新图像进行发送。
视频服务器上的视频应用接收到用户通过浏览器客户端的电脑发送的播放电影A的请求，然后该应用需要向该服务器本地的图像引擎发送渲染请求，使得图像引擎根据渲染请求渲染该电影A的画面，从而生成对应视频画面。然后，图形引擎向服务器本地的驱动器发送指令或命令，以使驱动器将该指令或命令转换成预置指令或命令，如qxl图像命令流的命令，该命令中具有对应的生成图像，即待更新图像。然后驱动器可以将该命令发送至命令队列中，以等待处理。由此可以从该命令中获取到对应的待更新图像。
视频服务器在接收到前文所述的qxl图像命令流的命令，该命令可以是由前文所述其它服务器发送过来的。视频服务器通过其中的模式转换器202接收到该命令，从该命令中可以获取到待更新图像。该命令中除了携带有该待更新图像，还可以携带有该图像的信息，如图像的尺寸，图像的长宽，然后模式转换器202根据该尺寸确定该面积。当该面积大于面积阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。
此外，模式转换器202还可以根据待更新图像的面积以及展示屏幕的面积，确定待更新图像在展示屏幕上的比例。当该比例大于比例阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。就不再过多赘述了。
此外，模式转换器202还可以确定或统计在预置刷新周期内，多个待更新图像的面积之和。当该面积之和大于面积之和阈值，则确定处理方式为视频流的处理方式。如果当前的处理方式已经是视频流处理方式则无需更改，如果当前的处理方式是图形命令流的处理方式，则需要进行处理方式的切换，将图形命令流的处理方式切换为视频流的处 理方式。否则确定处理方式为图形命令流的处理方式，根据上述可知，如果需要进行处理方式的切换则进行切换，如果不需要维持处理方式即可。就不再过多赘述了。
视频服务器中的模式转换器202在确定了处理方式后，对待更新图像进行处理。当处理方式是图形命令流的处理方式，则模式转换器202，针对待更新图像进行图像压缩，如通过jpeg标准进行图像压缩，并生成对应的图形命令流203，即执行了步骤211：生成图形命令流。然后将图形命令流203通过上述传输协议发送至用户的电脑，即执行了步骤213：发送图形命令流，以进行更新展示。
当处理方式是视频流的处理方式，则模式转换器202，针对待更新图像进行视频编码，如通过mjpeg视频编码对待更新图像进行编码，并生成对应的视频流204，即执行了步骤212：生成视频流。然后将视频流204通过上述传输协议发送至用户的电脑，即执行了步骤214：发送视频流，以进行更新展示。上述发送方式可以是通过协议包进行发送的。
在确定是图形命令流的处理方式后，针对待更新图像进行jpeg标准压缩，当完成jpeg标准压缩后，统计在上述一个刷新周期内所有进行图形命令流的处理方式压缩后的待更新图像的大小，如果超出网络的带宽阈值，则放弃图形命令流的处理方式的压缩后的待更新图像，并将处理方式进行换到视频流的处理方式，进行待更新图像的视频编码。即可以重新通过上述qxl图像命令流的命令获取待更新图像然后进行视频编码，以进行传输。
浏览器客户端收到协议数据后，除了上文所述的图像更新外，还可以根据待更新图像的生成时间和最后在客户端上的显示时间进行时延的统计，此外再将收到的协议包的大小进行统计，统计网络的带宽数据大小，当统计后的更新时延和/或统计的带宽数据超出对应的阈值时，则生成QOE(Quality of Experience，体验质量)的反馈消息，反馈传递给视频服务器，该反馈信息可以表示为切换到视频流的处理方式。使得视频服务器中的模式转换器202执行了步骤215：接收反馈信息，然后根据该反馈信息将处理方式切换到视频流的处理方式。即当前的处理方式已经属于图形命令流的处理方式时，如果收到反馈信息，则切换成视频流的处理方式，如果当前已经属于视频流的处理方式，则更新接收到反馈信息的时间，即更新了切换到视频流的处理方式的时间(即使当前已经是视频流的处理方式了，也依旧作为更新处理方式的最新时间)。
当当前的处理方式是视频流的处理方式的时候，且如果在一定时间或预置刷新时间内都未接收到反馈信息，则退出视频流的处理方式，切换到图形命令流的处理方式进行对待更新图像的处理。
此处未详细记载的内容可以参考前文所述的内容，就不再赘述。
在上述本实施例中，第一设备301、第二设备302进行网络连接。若第一设备301、第二设备302是通信连接，该移动网络的网络制式可以为2G(GSM)、2.5G(GPRS)、 3G(WCDMA、TD-SCDMA、CDMA2000、UTMS)、4G(LTE)、4G+(LTE+)、WiMax、5G等中的任意一种。
根据相似的发明构思，本申请一示例性实施例的一种媒体信息的传输方法。本申请实施例提供的该方法由计算设备执行，如，电脑等。如图4所示，该方法400包括以下步骤：
401：接收处理后的待更新图像，并根据处理后的图像得到待更新图像。
402：将待更新图像在展示屏幕上进行更新。
403：确定待更新图像在展示屏幕上的更新时间与待更新图像的生成时间之间的时间差。
404：当时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到反馈信息后，根据反馈信息确定待更新图像的处理方式为视频流的处理方式。
需要说明的是，由于步骤401-404的具体实施方式与前文阐述的方法100的具体实施方式相似，这里就不再赘述。仅说明：发送端可以是指前文中所述的服务器。此外，该执行主体也可以是其它设备，如服务器等其它交互设备。
该方法400未能详尽的内容请参考前文，就不再赘述。
图5为本申请一示例性实施例提供的一种媒体信息的传输装置的结构框架示意图。该装置500可以应用于服务器。该装置500包括：获取模块501、确定模块502、处理模块503；以下针对各个模块的功能进行详细的阐述：
获取模块501，用于获取待更新图像。
确定模块502，用于根据待更新图像，确定图像的处理方式，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景。
处理模块503，用于根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
具体的，确定模块502，用于根据待更新图像的尺寸、待更新图像在展示场景中的更新时间和/或处理后的待更新图像所占用的带宽数据，确定处理方式为图像命令流的处理方式或视频流的处理方式。
具体的，获取模块501，用于接收图像指令，从指令中获取待更新图像。
具体的，确定模块502，包括：第一确定单元，用于确定待更新图像的面积；确定面积是否大于面积阈值，当小于或等于面积阈值，则确定处理方式为图形命令流的处理方式；当大于所述面积阈值，则确定处理方式为视频流的处理方式。
具体的，确定模块502，包括：第二确定单元，用于确定待更新图像的面积，并确定面积在展示屏幕中的比例；确定比例是否大于比例阈值，当小于或等于比例阈值，则确定处理方式为图形命令流的处理方式；当大于比例阈值，则确定处理方式为视频流的 处理方式。
具体的，确定模块502，包括：第三确定单元，用于在预置时间内，确定多个待更新图像对应的面积，并确定多个待更新图像的面积之和；确定面积之和是否大于面积之和的阈值，当小于或等于面积之和的阈值，则确定处理方式为图形命令流的处理方式；当大于面积之和的阈值，则确定处理方式为视频流的处理方式。
具体的，处理模块503，包括：第一处理单元，用于当确定的处理方式为图形命令流的处理方式，则根据图形命令流的压缩方式，对待更新图像进行压缩，将压缩后的待更新图像进行传输；第二处理单元，用于当确定的处理方式为视频流的处理方式，则根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
此外，在根据图形命令流的压缩方式，对待更新图像进行压缩后，确定模块502，还用于在预置时间内，确定多个压缩后的待更新图像的大小之和；该装置500还包括：传输模块，用于当大小之和小于或等于大小之和的阈值，则将当前压缩后的待更新图像进行传输；确定模块502，还用于当大小之和大于大小之和的阈值，将停止传输当前压缩后的待更新图像，并确定当前待更新图像的处理方式为视频流的处理方式；获取模块501，还用于获取待更新图像，根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
此外，确定模块502，还用于接收接收端发送的反馈信息，根据反馈信息确定处理方式为视频流的处理方式，其中接收端是根据接收到的待更新图像的更新时延和/或处理后的待更新图像所占用的带宽数据发送的反馈信息。
此外，确定模块502，还用于在预置时间内，未接收到反馈信息，如果当前的处理方式为视频流的处理方式，则将处理方式切换为图形命令流的处理方式。
本装置500中未能详尽的内容请参考前文所述，就不再赘述。
图6为本申请一示例性实施例提供的一种媒体信息的传输装置的结构框架示意图。该装置600可以应用于终端设备。该装置600包括：接收模块601、更新模块602、确定模块603以及发送模块604；以下针对各个模块的功能进行详细的阐述：
接收模块601，用于接收处理后的待更新图像，并根据处理后的图像得到待更新图像。
更新模块602，用于将待更新图像在展示屏幕上进行更新。
确定模块603，用于确定待更新图像在展示屏幕上的更新时间与待更新图像的生成时间之间的时间差。
发送模块604，用于当时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到所述反馈信息后，根据反馈信息确定待更新图像的处理方式为视频流的处理方式。
本装置600中未能详尽的内容请参考前文所述，就不再赘述。
以上描述了图5所示的装置500的内部功能和结构，在一个可能的设计中，图5所示的装置500的结构可实现为计算设备，如，服务器。如图7所示，该设备700可以包括：存储器701、处理器702；
存储器701，用于存储计算机程序。
处理器702，用于执行计算机程序，以用于：获取待更新图像；根据所述待更新图像，确定图像的处理方式，处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
具体的，处理器702，具体用于：根据待更新图像的尺寸、待更新图像在展示场景中的更新时间和/或处理后的待更新图像所占用的带宽数据，确定处理方式为图像命令流的处理方式或视频流的处理方式。
具体的，处理器702，具体用于：接收图像指令，从指令中获取待更新图像。
具体的，处理器702，具体用于确定待更新图像的面积；确定面积是否大于面积阈值，当小于或等于面积阈值，则确定处理方式为图形命令流的处理方式；当大于所述面积阈值，则确定处理方式为视频流的处理方式。
具体的，处理器702，具体用于：确定待更新图像的面积，并确定面积在展示屏幕中的比例；确定比例是否大于比例阈值，当小于或等于比例阈值，则确定处理方式为图形命令流的处理方式；当大于比例阈值，则确定处理方式为视频流的处理方式。
具体的，处理器702，具体用于：在预置时间内，确定多个待更新图像对应的面积，并确定多个待更新图像的面积之和；确定面积之和是否大于面积之和的阈值，当小于或等于面积之和的阈值，则确定处理方式为图形命令流的处理方式；当大于面积之和的阈值，则确定处理方式为视频流的处理方式。
具体的，处理器702，具体用于：当确定的处理方式为图形命令流的处理方式，则根据图形命令流的压缩方式，对待更新图像进行压缩，将压缩后的待更新图像进行传输；当确定的处理方式为视频流的处理方式，则根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
此外，在根据图形命令流的压缩方式，对待更新图像进行压缩后，处理器702，还用于：在预置时间内，确定多个压缩后的待更新图像的大小之和；当大小之和小于或等于大小之和的阈值，则将当前压缩后的待更新图像进行传输；当大小之和大于大小之和的阈值，将停止传输当前压缩后的待更新图像，并确定当前待更新图像的处理方式为视频流的处理方式；获取待更新图像，根据视频流的编码方式，对待更新图像进行编码，将编码后的待更新图像进行传输。
此外，该设备700还可以包括通信组件，通信组件，用于：接收接收端发送的反馈信息，根据反馈信息确定处理方式为视频流的处理方式，其中接收端是根据接收到的待 更新图像的更新时延和/或处理后的待更新图像所占用的带宽数据发送的反馈信息。
此外，处理器702，还用于：在预置时间内，未接收到反馈信息，如果当前的处理方式为视频流的处理方式，则将处理方式切换为图形命令流的处理方式。
另外，本发明实施例提供了一种计算机存储介质，计算机程序被一个或多个处理器执行时，致使一个或多个处理器实现图1-2方法实施例中一种媒体信息的传输方法的步骤。
以上描述了图6所示的装置600的内部功能和结构，在一个可能的设计中，图6所示的装置600的结构可实现为计算设备，如，电脑。如图8所示，该设备800可以包括：存储器801、处理器802以及通信组件803；
存储器801，用于存储计算机程序。
通信组件803，用于接收处理后的待更新图像。
处理器802，用于执行计算机程序，以用于：根据处理后的图像得到待更新图像；将待更新图像在展示屏幕上进行更新；确定待更新图像在展示屏幕上的更新时间与待更新图像的生成时间之间的时间差。
通信组件803，用于当时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到反馈信息后，根据反馈信息确定待更新图像的处理方式为视频流的处理方式。
另外，本发明实施例提供了一种计算机存储介质，计算机程序被一个或多个处理器执行时，致使一个或多个处理器实现图4方法实施例中一种媒体信息的传输方法的步骤。
另外，在上述实施例及附图中的描述的一些流程中，包含了按照特定顺序出现的多个操作，但是应该清楚了解，这些操作可以不按照其在本文中出现的顺序来执行或并行执行，操作的序号如101、102、103等，仅仅是用于区分开各个不同的操作，序号本身不代表任何的执行顺序。另外，这些流程可以包括更多或更少的操作，并且这些操作可以按顺序执行或并行执行。需要说明的是，本文中的“第一”、“第二”等描述，是用于区分不同的消息、设备、模块等，不代表先后顺序，也不限定“第一”和“第二”是不同的类型。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助加必需的通用硬件平台的方式来实现，当然也可以通过硬件和软件结合的方式来实现。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以计算机 产品的形式体现出来，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程多媒体数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程多媒体数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程多媒体数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程多媒体数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然 可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (14)

  1. 一种媒体信息的传输方法，其特征在于，包括：
    获取待更新图像；
    根据所述待更新图像，确定图像的处理方式，所述处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；
    根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的图像，进行展示。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述待更新图像，确定图像的处理方式，包括：
    根据所述待更新图像的尺寸、待更新图像在展示场景中的更新时间和/或处理后的待更新图像所占用的带宽数据，确定处理方式为图像命令流的处理方式或视频流的处理方式。
  3. 根据权利要求1所述的方法，其特征在于，所述获取待更新图像，包括：
    接收图像指令，从所述指令中获取待更新图像。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述待更新图像，确定图像的处理方式，包括：
    确定所述待更新图像的面积；
    确定所述面积是否大于面积阈值，当小于或等于面积阈值，则确定所述处理方式为图形命令流的处理方式；
    当大于所述面积阈值，则确定所述处理方式为视频流的处理方式。
  5. 根据权利要求1或4所述的方法，其特征在于，所述根据所述待更新图像，确定图像的处理方式，包括：
    确定所述待更新图像的面积，并确定所述面积在展示屏幕中的比例；
    确定所述比例是否大于比例阈值，当小于或等于比例阈值，则确定所述处理方式为图形命令流的处理方式；
    当大于所述比例阈值，则确定所述处理方式为视频流的处理方式。
  6. 根据权利要求1或5所述的方法，其特征在于，所述根据所述待更新图像，确定图像的处理方式，包括：
    在预置时间内，确定多个待更新图像对应的面积，并确定多个待更新图像的面积之和；
    确定所述面积之和是否大于面积之和的阈值，当小于或等于面积之和的阈值，则确定所述处理方式为图形命令流的处理方式；
    当大于所述面积之和的阈值，则确定所述处理方式为视频流的处理方式。
  7. 根据权利要求1所述的方法，其特征在于，所述根据确定的处理方式，对待更新图像进行处理并传输，包括：
    当确定的处理方式为图形命令流的处理方式，则根据图像命令流的压缩方式，对所述待更新图像进行压缩，将压缩后的待更新图像进行传输；
    当确定的处理方式为视频流的处理方式，则根据视频流的编码方式，对所述待更新图像进行编码，将编码后的待更新图像进行传输。
  8. 根据权利要求7所述的方法，其特征在于，在根据图形命令流的压缩方式，对所述待更新图像进行压缩后，所述方法还包括：
    在预置时间内，确定多个压缩后的待更新图像的大小之和；
    当所述大小之和小于或等于大小之和的阈值，则将当前压缩后的待更新图像进行传输；
    当所述大小之和大于大小之和的阈值，将停止传输当前压缩后的待更新图像，并确定当前待更新图像的处理方式为视频流的处理方式；
    获取待更新图像，根据视频流的编码方式，对所述待更新图像进行编码，将编码后的待更新图像进行传输。
  9. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    接收接收端发送的反馈信息，根据所述反馈信息确定所述处理方式为视频流的处理方式，其中所述接收端是根据接收到的待更新图像的更新时延和/或处理后的待更新图像所占用的带宽数据发送的所述反馈信息。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    在预置时间内，未接收到所述反馈信息，如果当前的处理方式为视频流的处理方式，则将所述处理方式切换为图形命令流的处理方式。
  11. 一种媒体信息的传输方法，其特征在于，包括：
    接收处理后的待更新图像，并根据处理后的图像得到待更新图像；
    将所述待更新图像在展示屏幕上进行更新；
    确定所述待更新图像在展示屏幕上的更新时间与所述待更新图像的生成时间之间的时间差；
    当所述时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到所述反馈信息后，根据所述反馈信息确定待更新图像的处理方式为视频流的处理方式。
  12. 一种计算设备，其特征在于，包括：存储器、处理器；
    所述存储器，用于存储计算机程序；
    所述处理器，执行所述计算机程序，以用于：
    获取待更新图像；
    根据所述待更新图像，确定图像的处理方式，所述处理方式使得处理后的待更新图像满足传输条件和/或图像的展示场景；
    根据确定的处理方式，对待更新图像进行处理并传输，以使接收端接收到处理后的 图像，进行展示。
  13. 一种计算设备，其特征在于，包括：存储器、处理器以及通信组件；
    所述存储器，用于存储计算机程序；
    所述通信组件，用于接收处理后的待更新图像；
    所述处理器，执行所述计算机程序，以用于：根据处理后的图像得到待更新图像；
    将所述待更新图像在展示屏幕上进行更新；
    确定所述待更新图像在展示屏幕上的更新时间与所述待更新图像的生成时间之间的时间差；
    所述通信组件，用于当所述时间差大于时间阈值和/或处理后的待更新图像所占用的带宽大于带宽阈值，发送反馈信息至发送端，以使发送端接收到所述反馈信息后，根据所述反馈信息确定待更新图像的处理方式为视频流的处理方式。
  14. 一种存储有计算机程序的计算机可读存储介质，其特征在于，计算机程序被一个或多个处理器执行时，致使所述一个或多个处理器实现权利要求1-11任一项所述方法中的步骤。
PCT/CN2022/118422 2021-09-18 2022-09-13 媒体信息的传输方法、计算设备及存储介质 WO2023040825A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111101871.9A CN113973224B (zh) 2021-09-18 媒体信息的传输方法、计算设备及存储介质
CN202111101871.9 2021-09-18

Publications (1)

Publication Number Publication Date
WO2023040825A1 true WO2023040825A1 (zh) 2023-03-23

Family

ID=79586658

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/118422 WO2023040825A1 (zh) 2021-09-18 2022-09-13 媒体信息的传输方法、计算设备及存储介质

Country Status (1)

Country Link
WO (1) WO2023040825A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100209011A1 (en) * 2009-02-13 2010-08-19 Fuji Xerox Co., Ltd. Image communication apparatus and computer readable medium for controlling image communication
CN103796012A (zh) * 2014-01-20 2014-05-14 北京航空航天大学 无人机多源异类侦察图像压缩接口适配方法
CN107659827A (zh) * 2017-09-25 2018-02-02 北京小鱼易连科技有限公司 基于内容分析的桌面视频编码控制系统
CN111010582A (zh) * 2019-12-18 2020-04-14 深信服科技股份有限公司 一种云桌面图像处理方法、装置、设备及可读存储介质
CN112672155A (zh) * 2020-12-18 2021-04-16 厦门亿联网络技术股份有限公司 一种基于共享类型判别的桌面共享方法、装置及存储介质
CN113973224A (zh) * 2021-09-18 2022-01-25 阿里巴巴（中国）有限公司 媒体信息的传输方法、计算设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100209011A1 (en) * 2009-02-13 2010-08-19 Fuji Xerox Co., Ltd. Image communication apparatus and computer readable medium for controlling image communication
CN103796012A (zh) * 2014-01-20 2014-05-14 北京航空航天大学 无人机多源异类侦察图像压缩接口适配方法
CN107659827A (zh) * 2017-09-25 2018-02-02 北京小鱼易连科技有限公司 基于内容分析的桌面视频编码控制系统
CN111010582A (zh) * 2019-12-18 2020-04-14 深信服科技股份有限公司 一种云桌面图像处理方法、装置、设备及可读存储介质
CN112672155A (zh) * 2020-12-18 2021-04-16 厦门亿联网络技术股份有限公司 一种基于共享类型判别的桌面共享方法、装置及存储介质
CN113973224A (zh) * 2021-09-18 2022-01-25 阿里巴巴（中国）有限公司 媒体信息的传输方法、计算设备及存储介质

Also Published As

Publication number Publication date
CN113973224A (zh) 2022-01-25

Similar Documents

Publication Publication Date Title
US20220038724A1 (en) Video stream decoding method and apparatus, terminal device, and storage medium
US20220263885A1 (en) Adaptive media streaming method and apparatus according to decoding performance
CN111399964B (zh) 一种基于视频串流技术的云桌面平台
WO2017219896A1 (zh) 视频流的传输方法及装置
US20160323547A1 (en) High Quality Multimedia Transmission from a Mobile Device for Live and On-Demand Viewing
JP5123186B2 (ja) 任意のフォーマットにおけるラージオブジェクトの通信のためのリモートプロトコルサポート
WO2017107911A1 (zh) 一种视频云平台播放视频的方法及装置
US9426476B2 (en) Video stream
US20140187331A1 (en) Latency reduction by sub-frame encoding and transmission
US10200707B2 (en) Video bit stream decoding
CN110582012B (zh) 视频切换方法、视频处理方法、装置及存储介质
US20180213274A1 (en) Apparatus and method for playing back and seeking media in web browser
AU2004305808A1 (en) Method and apparatus for generating graphical and media displays at a thin client
CN111221491A (zh) 交互控制方法及装置、电子设备、存储介质
CN110662100A (zh) 一种信息处理方法、装置、系统和计算机可读存储介质
JP6511038B2 (ja) クラウドストリーミングサービス提供方法及びそのための装置
JP7151004B2 (ja) 中断可能な映像トランスコーディング
KR20180086112A (ko) 웹 브라우저에서 미디어를 재생하고 탐색하는 장치 및 방법
US20120265858A1 (en) Streaming portions of a quilted graphic 2d image representation for rendering into a digital asset
CN114222156A (zh) 视频剪辑方法、装置、计算机设备和存储介质
WO2023040825A1 (zh) 媒体信息的传输方法、计算设备及存储介质
CN115941965A (zh) 云桌面编码方法、重建方法、显示方法和显示系统
CN113973224B (zh) 媒体信息的传输方法、计算设备及存储介质
CN116636224A (zh) 可替代联网应用程序服务的系统和方法
CN117319592B (zh) 一种云桌面摄像头重定向方法、系统及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22869194

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE