WO2021139418A1 - Appareil de traitement d'image, dispositif distant et système de communication - Google Patents

Appareil de traitement d'image, dispositif distant et système de communication Download PDF

Info

Publication number
WO2021139418A1
WO2021139418A1 PCT/CN2020/130436 CN2020130436W WO2021139418A1 WO 2021139418 A1 WO2021139418 A1 WO 2021139418A1 CN 2020130436 W CN2020130436 W CN 2020130436W WO 2021139418 A1 WO2021139418 A1 WO 2021139418A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
unit
control
mouse
image
Prior art date
Application number
PCT/CN2020/130436
Other languages
English (en)
Chinese (zh)
Inventor
杨璐
范志刚
Original Assignee
西安万像电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西安万像电子科技有限公司 filed Critical 西安万像电子科技有限公司
Publication of WO2021139418A1 publication Critical patent/WO2021139418A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen

Definitions

  • the present disclosure relates to the field of computer coding, and in particular to image processing devices, remote equipment and communication systems.
  • the shortcomings of the remote picture transmission method based on drawing instructions include: First, in order to draw the display picture at the remote end, part of the source file data needs to be carried during transmission. However, since the remote end saves the active file data, there are security problems; Second, since the drawing commands between computers may be different, only the remote drawing command is the same as the drawing command at the source end can the remote end decode and display the picture. Therefore, the applicability of this method is poor.
  • the shortcomings of the remote picture transmission method based on pixel coding include: first, the quality of the displayed picture cannot be dynamically adjusted at the remote end; second, the commonly used coding protocol is H.264 or H.265 or other open source video protocols. The open source of the protocol may lead to lower network transmission security.
  • the embodiments of the present disclosure provide an image processing device, a remote device, and a communication system, which can implement dynamic adjustment of transmission parameters to transmit sound and image data in the communication system.
  • the technical solution is as follows:
  • an image processing device which includes: an image acquisition unit, an image encoding unit, a sound acquisition unit, a first sound encoding unit, and a first transmission control unit;
  • the image collection unit is used to collect the computer screen according to the preset first control parameter, the sound collection unit is used to parameter collect the first sound data, and the first control parameter is used to control the collection of the computer screen;
  • the image coding unit is used to perform image coding on the computer screen according to the preset second control parameters to obtain the first coded data; the first sound coding unit is used to perform voice coding on the first sound data to obtain the second coded data; Two control parameters are used to control the image coding quality;
  • the first transmission control unit sends the first coded data and the second coded data to the remote device according to a preset third control parameter, and the third control parameter is used to control the transmission quality of the first transmission control unit to the remote device .
  • the device further includes a control decoding unit and a first sound decoding unit;
  • the first transmission control unit receives the reverse control data sent by the remote device, sends the reverse control data to the control decoding unit for decoding, obtains the first decoded data, and sends the first decoded data to the host.
  • the first transmission control unit receives the second sound data sent by the remote device, sends the second sound data to the first sound decoding unit for decoding, obtains the second decoded data, and sends the second decoded data to the host.
  • the first transmission control unit receives the USB mouse operation code data sent by the remote device, and sends the USB mouse operation code data to the host.
  • the device further includes a scheduling unit, which respectively obtains the first control data of the first transmission control unit, the second control data of the image encoding unit, and the third control data of the decoding unit, and is based on the first control data.
  • the control data, the second control data and the third control data update the first control parameter, the second control parameter and the third control parameter according to the preset decision model.
  • the first transmission control unit sequentially transmits the first decoded data, the second encoded data, and the second decoded data according to the preset priority.
  • a remote device in a second aspect of the embodiments of the present disclosure, includes: a second sound decoding unit and an image decoding unit, a second transmission control unit, and a playback control unit,
  • the second transmission control unit receives the first encoded data and the second encoded data sent by the image processing device, and sends the first encoded data to the image decoding unit to obtain the third decoded data, and sends the second encoded data to the second sound decoding Unit to obtain the fourth decoded data, and send the third decoded data and the fourth decoded data to the playback control unit;
  • the playback control unit controls the third decoded data and the fourth decoded data to perform synchronous display and playback.
  • the remote device further includes a USB unit, a mouse decoding unit, and a mouse drawing unit.
  • the second transmission control unit also receives mouse graphic data generated by user operations, and sends the mouse graphic data to the mouse decoding unit for decoding.
  • the control decoded data is sent to the mouse drawing unit; the USB unit obtains the USB mouse operation data and sends the USB mouse operation data to the mouse drawing unit; the mouse drawing unit is used to draw the mouse according to the control decoded data and the mouse operation data.
  • the remote device further includes a keyboard and mouse encoding unit.
  • the USB unit is also used to send USB mouse operation data to the keyboard and mouse encoding unit.
  • the keyboard and mouse encoding unit encodes and compresses the USB mouse operation data to obtain the USB mouse operation. Encoding data, and sending the USB mouse operation encoded data to the image processing device through the second transmission control unit.
  • the second transmission control unit receives the control data sent by the image processing device, and sends the control data to the mouse decoding unit.
  • the mouse decoding unit sends the decoded control data to the playback control unit, and the playback control unit according to The control data displays the corresponding mouse graphics.
  • a communication system which includes a host, the image processing apparatus disclosed in the above-mentioned first aspect, and the remote device disclosed in the above-mentioned second aspect.
  • the embodiments of the present disclosure can realize the transmission of sound and image data by dynamically adjusting transmission parameters in the communication system. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.
  • Fig. 1 is a structural diagram of an image processing device provided by an embodiment of the present disclosure
  • Figure 2 is a structural diagram of an image processing device provided by an embodiment of the present disclosure
  • Fig. 3 is a structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • Figure 4 is a structural diagram of a remote device provided by an embodiment of the present disclosure.
  • FIG. 5 is a structural diagram of a remote device provided by an embodiment of the present disclosure.
  • Figure 6 is a structural diagram of a remote device provided by an embodiment of the present disclosure.
  • FIG. 7 is a structural diagram of a communication system provided by an embodiment of the present disclosure.
  • FIG. 8 is a structural diagram of a host provided by an embodiment of the present disclosure.
  • FIG. 9 is a schematic diagram of the relationship between systems provided by an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • Fig. 11 is a work flow of an image coding unit provided by an embodiment of the present disclosure.
  • FIG. 12 is a schematic diagram of scheduling input and output provided by an embodiment of the present disclosure.
  • FIG. 13 is a schematic diagram of the relationship between transmission code rate, time delay and display quality provided by an embodiment of the present disclosure
  • FIG. 14 is a structural diagram of a transmission control unit provided by an embodiment of the present disclosure.
  • FIG. 15 is a schematic diagram of a redundant generation model provided by an embodiment of the present disclosure.
  • FIG. 16 is a schematic diagram of a prediction mechanism model provided by an embodiment of the present disclosure.
  • FIG. 17 is a schematic diagram of a prediction mechanism model provided by an embodiment of the present disclosure.
  • FIG. 1 is an image processing device provided by an embodiment of the present disclosure.
  • the image processing device 200 shown in FIG. 1 includes: an image acquisition unit 201, an image encoding unit 205, a sound acquisition unit 202, a first sound encoding unit 206, and a first sound encoding unit 206.
  • the image collection unit 201 is used to collect the computer screen according to the preset first control parameter
  • the sound collection unit 202 is used to collect the first sound data by the parameter
  • the first control parameter is used to control the collection of the computer screen
  • the image encoding unit 205 is used to perform image encoding on the computer screen according to the preset second control parameters to obtain the first encoded data; the first audio encoding unit 206 is used to perform voice encoding on the first sound data to obtain the second encoded data ;
  • the second control parameter is used to control the image coding quality;
  • the first transmission control unit 209 sends the first coded data and the second coded data to the remote device according to a preset third control parameter, and the third control parameter is used to control the transmission of the first transmission control unit to the remote device quality.
  • FIG. 2 is an image processing device provided by an embodiment of the present disclosure.
  • the image processing device 200 shown in FIG. 2 further includes: a control decoding unit 207 and a first sound decoding unit 208;
  • the first transmission control unit 209 receives the reverse control data sent by the remote device, sends the reverse control data to the control decoding unit for decoding, obtains the first decoded data, and sends the first decoded data to the host.
  • the first transmission control unit 209 receives the second sound data sent by the remote device, sends the second sound data to the first sound decoding unit 208 for decoding, obtains the second decoded data, and sends the second decoded data to the host.
  • the first transmission control unit 209 also receives the USB mouse operation coded data sent by the remote device, and sends the USB mouse operation coded data to the host.
  • FIG. 3 is an image processing device provided by an embodiment of the present disclosure.
  • the image processing device 200 shown in FIG. 3 further includes: a scheduling unit 210.
  • the scheduling unit 210 obtains the first control data and the image of the first transmission control unit 209, respectively.
  • the first transmission control unit 209 preferentially transmits the first decoded data, the second encoded data, and the second decoded data.
  • FIG. 4 is a remote device provided by an embodiment of the present disclosure.
  • the remote device 300 shown in FIG. 4 includes: a second sound decoding unit 304, an image decoding unit 303, a second transmission control unit 308, and a playback control unit 302 ,
  • the second transmission control unit 308 receives the first encoded data and the second encoded data sent by the image processing device, and sends the first encoded data to the image decoding unit 303 to obtain the third decoded data, and sends the second encoded data to the second
  • the sound decoding unit 304 obtains the fourth decoded data, and sends the third decoded data and the fourth decoded data to the playback control unit 302;
  • the playback control unit controls 302 to perform synchronous display and playback according to the third decoded data and the fourth decoded data.
  • FIG. 5 is a remote device provided by an embodiment of the present disclosure.
  • the remote device 300 shown in FIG. 5 further includes: a USB unit 306, a mouse decoding unit 311, and a mouse drawing unit and a mouse drawing unit 310.
  • the second transmission control unit 308 also receives the mouse graphic data generated by the user operation, and sends the mouse graphic data to the mouse decoding unit 311 for decoding to obtain control decoding data, and sends the control decoding data to the mouse drawing unit mouse drawing unit 310; USB unit 306 obtains the USB mouse operation data, and sends the USB mouse operation data to the mouse drawing unit mouse drawing unit 310; the mouse drawing unit mouse drawing unit 310 is used to draw the mouse according to the control decoding data and the mouse operation data.
  • FIG. 6 is a remote device provided by an embodiment of the present disclosure.
  • the remote device 300 shown in FIG. 6 further includes: a keyboard and mouse encoding unit 305,
  • the USB unit 306 is also used to send the USB mouse operation data to the keyboard and mouse encoding unit 305.
  • the keyboard and mouse encoding unit 305 encodes and compresses the USB mouse operation data to obtain the USB mouse operation encoding data, and passes the USB mouse operation encoding data through the second
  • the transmission control unit 308 sends to the image processing device.
  • the second transmission control unit 308 also receives the control data sent by the image processing device, and sends the control data to the mouse decoding unit 311.
  • the mouse decoding unit 311 sends the decoded control data to the playback control unit 302, and the playback The control unit 302 displays the corresponding mouse graphics according to the control data.
  • the second transmission control unit 308 receives the data sent from the network, and sends it to the image decoding unit 303, and the second sound decoding unit 304 to the playback control unit 302.
  • the playback control unit 302 decides when to send it to the display Display or play in the display unit 301; 308 also receives the mouse graphic data generated by the user operation, sends it to the mouse graphic or animation decoding 311 for decoding, and then sends it to the mouse to draw the mouse drawing unit 310; the mouse drawing unit 310 also receives the second Mouse operation data input by the USB unit 306.
  • the instructions for mouse operation are as follows:
  • the user performs mouse operations on the remote device 300, and the remote device 300 obtains the user's mouse operation data through the second USB unit 306;
  • the second USB unit 306 reports the mouse operation data to the mouse drawing unit and the mouse drawing unit 310 confirms the coordinates for use in mouse drawing; at the same time, the second USB unit 306 also sends the mouse operation data to the mouse and keyboard encoding unit 305 , The mouse operation data is encoded and compressed by the mouse and keyboard encoding unit 305 and then sent to the second transmission control unit 308, and then sent back to the image processing device 200 via the network card 309 and the network card 109, and the mouse operation data is sent to the control input unit 203
  • the operating system of the host 100 that is, the control input unit 203 is equivalent to an analog keyboard and mouse;
  • the host 100 executes the corresponding actions according to the mouse operation data, and generates the displayed images after the actions are executed.
  • the image processing device 200 collects and encodes the displayed images, and sends the encoded images through the network card 109 and the network card 309 To the second transmission control unit 308; in addition, the image processing device 200 also needs to determine whether the mouse graphics corresponding to the current mouse operation data (such as arrows, hand shapes, vertical bars, etc.) appear for the first time, and if so, then image processing The device 200 also encodes the mouse image data and sets an identifier corresponding to the mouse image data, and sends the encoded mouse image data and the identifier corresponding to the mouse image to the second transmission control unit 308 via the network card 109 and the network card 309, At the same time, the corresponding relationship between the mouse image and the identifier is saved. If not, the image processing device 200 can send the identifier corresponding to the mouse image to the second transmission control unit 308 via the network card 109 and the network card 309.
  • the second transmission control unit 308 receives the encoded image, encoded mouse graphic data and identifier, or identifier sent from the host 100, and sends the encoded image to the image decoding unit 303 for decoding, and the encoded mouse The graphic data and the identifier, or the identifier are sent to the mouse decoding unit 311 for decoding.
  • the mouse decoding unit 311 receives the coded mouse graphic data and the identifier, the corresponding relationship between the mouse graphic and the identifier (specifically the corresponding relationship between the mouse image information and the identifier) is saved, and the coded mouse graphic data is decoded Send to the playback control unit 302; the mouse drawing unit The mouse drawing unit 310 sends the mouse coordinates to the playback control unit 302, and the playback control unit 302 displays on the display unit display unit 301 according to the image, the mouse image, and the mouse coordinates.
  • the mouse decoding unit 311 If the mouse decoding unit 311 receives the identifier, it determines the mouse graphic information corresponding to the identifier according to the identifier and the previously saved correspondence, and sends the mouse graphic information to the mouse drawing unit 310; the mouse drawing unit 310 draws a mouse image according to the mouse graphic information. At the same time, the mouse drawing unit 310 stores the mouse coordinates, and the mouse drawing unit 310 displays the image, the mouse image, and the mouse coordinates on the display unit 301.
  • the remote device 300 already knows the complete set of mouse images when the connection between the remote device 300 and the host 100 is established, then even if the mouse image in the host 100 changes for the first time, there is no need to send the mouse image again. Just send the identifier corresponding to the mouse image directly.
  • the second transmission control unit 308 will also receive the audio input data of the sound input 307, encode and compress it in the sound encoding device 312, and then send it to 308.
  • the encoded audio will be sent to the 200 via the network card 109 and the network card 309 by 308. Will be sent to 100's operating system, of which 204 is equivalent to simulating an audio input device.
  • the remote device 300 can be realized by independent hardware, or it can be realized by software integrated with other operating systems; 300 has a second transmission control unit 308, which has the same capabilities as the first transmission control unit 209, and the main function of the second transmission control unit 308 The function is to receive data, group packets and report, and then perform necessary signaling feedback.
  • the remote device 300 also has a sound input unit 307, an input sound encoding unit 312, which is transmitted back to the host 100 through the second transmission control unit 308, and finally input to the operating system through 108.
  • the remote device 300 also has a USB input unit 306.
  • the input here consists of two parts, one is the keyboard and mouse input, this part of the data is encoded by the keyboard and mouse encoding unit 305, and then transmitted back to 100 through 308, and finally input through 107 To the operating system.
  • Another type of data is when the image processing device 200 is integrated into the host 100, the second transmission control unit 308 and the first transmission control unit 209 will directly redirect the second USB unit 306 to the first USB 106, At this time, the operation on the second USB 306 is like directly plugging and removing devices such as a USB flash drive on the first USB 106.
  • the data received by the second transmission control unit 308 is divided into images and sounds.
  • mouse graphic data is also received.
  • These three kinds of data are respectively sent to the three decoding units for decoding, and then uniformly sent to the playback control unit 302, which is synchronized by 302, and the most important is the audio and video synchronization.
  • the playback control unit 302 determines the display unit of each hardware device. Played on 301.
  • the image capture unit 201 can choose to only capture the computer desktop instead of the mouse when capturing images; the mouse is drawn by the remote device 300 by itself, and the host 100 is in the process of establishing a connection with the remote device 300
  • the entire set of mouse images and animations are transmitted to the remote device 300, and the host 100 and the remote device 300 only transmit key values and mouse position information during the keyboard and mouse interaction; the remote device 300 will first perform mouse drawing.
  • the mouse zoom out drawing can reduce the operation delay and improve the user experience.
  • Fig. 7 is a structural diagram of a communication system provided by an embodiment of the present disclosure.
  • the communication system shown in Fig. 7 includes a host 100, any image processing apparatus 200 of the above-mentioned embodiments, and any one of the above-mentioned remote devices 300.
  • the host 100 represents a computer system at the source end, and the image processing apparatus 200 and the remote device 300 communicate through a network (WIFI, wired, 4G, etc.) of any medium.
  • the image processing apparatus 200 may be integrated in the host 100 for use, or may be used as an independent system independent of the host 100.
  • FIG. 8 is a schematic structural diagram of a host provided by an embodiment of the present disclosure.
  • the host as shown in FIG. 8 includes an image processing device,
  • Source 100 The computer-generated picture and sound data are sent to the image processing unit 104 and the sound processing unit 105 for processing.
  • the image processing device 200 obtains these data through the image acquisition unit 201 and the sound acquisition unit 202, and sends them to the image encoding unit.
  • the first sound encoding unit 206 performs encoding, and then send the encoding result to the first transmission control unit 209, which is sent to the remote device by the first transmission control unit 209; the first transmission control unit 209 simultaneously receives and sends from the remote device
  • the incoming user’s reverse control data and sound data are respectively sent to the first sound decoding unit 208 and the control decoding unit 207 for decoding, and the decoding results are sent to the sound input unit 108 and control input unit 107 of the computer system; the first transmission The control unit 209 may also receive the USB redirection data sent from the remote end, and directly send it to the first USB unit 106 of the system.
  • FIG. 9 Essentially, the relationship between the embodiment of the present disclosure and the computer system is shown in FIG. 9.
  • FIG. 10 is a schematic diagram of the structure of an image processing device (that is, the sending end of the image transmission system) provided by an embodiment of the present disclosure.
  • the image processing device 0 is divided into 5 units, collecting, encoding, transmitting, scheduling, and input, as shown in FIG. 10, It can be integrated in the source 100 for implementation, or can be used as an independent system; it can be implemented in software, or in hardware such as FPGA or chip.
  • the collection will collect picture pixel data and sound data at the same time.
  • the working position of the image acquisition unit 201 is different. If 200 is integrated within 100, 104 will send a copy of data to 201 when generating image data and writing it into the image cache. 201 can also obtain drawing instructions, image change data, and image movement data used when generating image data from 104. , These instructions and data can assist the coding unit to accelerate coding;
  • 201 obtains image data through the 100 physical image output interface, and can no longer obtain drawing instructions, image change data, and image movement data.
  • the workflow of the 201 image coding unit is shown in Figure 11. After receiving the data, the image coding unit first determines whether there is auxiliary data. If there is auxiliary data, analyze the auxiliary data, and then use the result directly, which can save time and reduce delay.
  • image recognition is required.
  • the basic rule of recognition is to compare the successive two frames of image data pixel by pixel, analyze the characteristics, and then divide the image into different types of macroblocks, and analyze the motion vector within the tile. .
  • the motion vector refers to the direction and distance the block moves within the frame.
  • the motion vector is processed first. If it is a motion vector, it is judged whether the current motion block is sent to the highest quality level. If it is reached, only the motion vector data is generated and sent to the next level to be packaged for transmission. If it is not reached, Then obtain the next quality layer data of the current block, and send it to the next level together with the motion vector for packing. If there is no motion in the current block, different types of image coding units are sent according to different types, including but not limited to text, picture, video and other coding units.
  • the image coding in the present invention is a multi-level or progressive coding technology.
  • the display may be blurred.
  • Each additional layer can be decoded to make it clearer.
  • the display quality of the original image can be achieved, that is, the highest quality Floor.
  • an image will be generated, which can be used for display, but which layer of the generated image requires the basic quality layer (the text marked with green background below this is introduced) to prevent the picture from being clear and unclear. Fluctuating back and forth.
  • the encoding unit will output the data that reaches the basic quality layer.
  • the subsequent quality layer data will be sent according to whether the current block position has changed. If there is no change, an extra layer (up to the highest quality layer) will be sent, and if there is a change, it will be re-encoded and sent to the basic Quality layer.
  • the image coding unit can divide the original image data into small image blocks after recognition. These small blocks are divided into 5 categories: text, picture, video, motion vector, and no change. Text, pictures, and videos all support the above-mentioned multi-quality layer (progressive) encoding.
  • these coding units are lossy and some are lossless. They are related to the characteristics of the image and the perception of the human eye. For example, video uses lossy coding, because the video is always moving, and appropriate lossy coding is subjective to people. There is no effect on the experience, and at the same time it can reduce the code stream and increase the encoding speed. At the same time, these coding units support multi-quality layer output, and the coding unit always outputs basic quality layer data. For example, a piece of image data is encoded into 10 layers, the basic quality layer may be 3. The coding unit only needs to output the data of quality layers 1-3, and the remaining data is kept in the context. The basic quality layer of the coding unit is determined by the scheduling control unit 210 according to the current Determined by the collected system parameters, each coding unit can use a different basic quality layer.
  • the image data that is unchanged data, which means that compared with the previous frame, this part of the data is judged whether it reaches the highest quality layer before each output, and if it is not reached, the next quality is output.
  • the data of the layer is used to pack the data to be sent. If it is reached, only the unchanged identifier needs to be sent.
  • the picture quality of the highest quality layer is determined by the maximum code rate.
  • Each coding unit determines the output code rate of its highest quality layer according to the proportion of blocks and the maximum code rate, thereby determining the output picture quality, and the maximum quality layer number of each coding unit It can also be different, determined by the coding unit itself, and recorded in the context of the coding unit.
  • the scheduling control unit 210 mainly collects the bandwidth, time delay, packet loss rate, packet error rate, and the next transmission network condition prediction reported from the first transmission control unit 209; 210 also collects the image coding unit 205 and the first voice coding The current operation time, output code rate, and output prediction of the next frame reported by the unit 206; the scheduling control unit 210 also collects mouse and keyboard control information for controlling the decoding unit 207.
  • the dispatch control unit 210 establishes a decision model based on the above data, outputs three sets of parameters, collects control parameters, encodes control parameters, and transmits control parameters.
  • the scheduling control unit 210 predicts based on the bandwidth, delay, packet loss rate, packet error rate, and the next transmission network condition, the current operation time of each coding unit, the output code rate, and the output prediction of the next frame, explain the forecast adjustment of the parameters in the next stage:
  • Example 1 The current bandwidth is 10M, the transmission delay is 2ms, the packet loss rate is 0.01%, and the packet error rate is 0.
  • the next transmission network condition is predicted to be in such a good state in the next stage, and the current acquisition frame rate is 5 frames, the current average encoding delay of the encoding unit is 10ms, the current output of the encoding unit is 2Mbps, and the next frame is expected to maintain 2Mbps without fluctuations in the code rate.
  • the current highest quality layer is 6, the basic quality layer is 3, and the user currently has keyboard and mouse operations. Behavior, and keyboard events take up more time.
  • the optimal decision output by the scheduling control unit 210 should be: the bandwidth is sufficient, the transmission is not restricted, and the packet loss rate is not high.
  • the user may be doing real-time document editing, and the delay requirement is high, which can be appropriately increased. Redundant quantity, if the proportion reaches 1% of the total data volume; the current user real-time text operation requires a higher impact on keyboard and mouse events, and the frame rate can be increased to 10 frames. The status of each unit in the next stage remains unchanged, and the frame can be increased.
  • the frame rate is 15 frames (the frame rate of text editing is as high as 20 frames, it is not a video, and the human eye will feel no difference); the text has a higher requirement for clarity, and the basic quality level is increased to 4, and the next stage is if each unit If the status remains unchanged, the basic quality layer can continue to be improved, and the highest quality layer will not exceed the maximum.
  • the coding unit does not need to limit the code rate, and the bandwidth of the next frame is sufficient, even if it exceeds 2Mbps.
  • Example 2 following the above scenario, if the user no longer edits the document and watches the video instead, at this time, the bandwidth is 10M, the transmission delay is 3ms, the packet loss rate is 0.01%, the packet error rate is 0, and the next transmission The network condition is predicted to be in such a good state in the next stage.
  • the current acquisition frame rate is 20fps (the above scene continues, the frame rate may reach the maximum frame rate under text operation, 20fps), and the current average encoding delay of the coding unit is 10ms, the current output of the coding unit is 5Mbps, the next frame is expected to have a possible code rate of 6-7Mbps, the current highest quality layer is 6, and the basic quality layer is 6 (the above scene continues, the basic quality layer may reach the highest quality layer) , And there is no keyboard and mouse operation behavior at this time.
  • the optimal decision output by the scheduling control unit 210 should be: According to the encoding unit recognition and collection recognition feedback, increase the collection frame rate, first to 25fps (up to 60fps, the current PC refresh frame rate is 60fps), because it reaches At 24fps, the human eye will not feel the picture intermittent, and the subsequent improvement can be continued; scheduling first reduces the basic quality layer of the coding unit, first reduces it to half of the original, and becomes 3; because at this time, the code stream can be reduced under natural video . Use a lossy video encoding unit, the code stream is limited to 5Mbps, and input the current frame rate of 25fps to the encoding unit (the subsequent acquisition frame rate needs to be increased by the scheduling control unit to continue to set this parameter), and the encoding unit will limit the code stream and frame rate Adjust the coding parameters to ensure that the average code stream fluctuates at 5Mbps; the 10M bandwidth is sufficient for transmission because there is no operation at this time, the delay requirement is reduced, and the amount of transmitted data
  • the amount of redundancy can be reduced. If the proportion reaches the total data amount 0.5 %. Other lost data is recovered by retransmission. Even if one frame is lost, the impact on video playback will not be too great.
  • the decision model in the scheduling control unit 210 may be based on a neural network model.
  • the input reference is input to the original neural network model, and the original neural network model is trained.
  • the model output result meets the conditions, the model After the training is completed, the trained neural network model is used as a decision model for practical application.
  • the network status may be different in each period of time. For example, a wide area network may have better quality in these two days, and the delay may decrease in a few days, but the packet loss may increase, etc. .
  • Training time continues to be calculated annually. For example, after one year of training, more than 80% of the network conditions can be encountered in the acquisition, encoding, decoding, and display performance of the machine unit, but there is no guarantee that all conditions can be encountered. Therefore, the decision model needs to be adjusted accordingly, that is to say, the prediction model needs to be continuously trained during the application process.
  • output parameters with good decision results under the current network state and corresponding input parameters can be recorded for training the decision model so that the decision model can be adjusted according to the current network state.
  • application networks can be divided into local area networks and wide area networks.
  • the conditions of the two networks are quite different.
  • two decision-making models can be trained for the local area network and the wide area network respectively.
  • the collection control parameters may affect the sampling frequency and output speed of the image collection unit 201 and the sound collection unit 202.
  • the encoding control parameter data affects the encoding quality of the image encoding unit 205, the first audio encoding unit 206, the operation time, and the output bit rate.
  • the scheduling control unit 210 may modify the basic quality layer to determine the display quality of the basic output picture, or it may modify the maximum output bit rate to affect the display quality when the picture is still, and it may also schedule and control the image coding.
  • the unit finds that the encoding time may be too long, and the network prediction can delay too long, thereby limiting the shortest running time of the image coding unit.
  • the image coding unit may adjust the coding parameters according to the running time to speed up the operation, and ensure The remote display output is smooth.
  • the scheduling input and output are shown in Figure 12.
  • the first transmission control unit 209 and the second transmission control unit 308 mainly implement functions such as data unpacking, sending, receiving, packet grouping, packet loss retransmission, delay control, and error correction control.
  • the structures of the first transmission control unit and the second transmission control unit are shown in FIG. 14.
  • the first transmission control unit 209 and the second transmission control unit 308 include a display protocol and a description map 2091, display all possible protocols generated by the upper layer, and generate a description content map for the data generated by the upper layer.
  • the description content includes the necessary attributes of the current data to Take image data as an example, such as frame number, frame type, quality level, etc.
  • the upper layer can divide various types of data according to the display protocol and description mapping 2091 into transmission weight priority and determine the queue for each data transmission according to the description, priority, and weight. Length and sending order.
  • the upper layer may correspond to the following data, images, sounds, keyboard and mouse operation data, file stream data, and other control or scheduling instructions.
  • the display protocol and description map 2091 are used to generate description content, that is, the necessary attributes of the current data. Taking image data as an example, such as frame number, frame type, quality level, etc.
  • Transmission queue scheduling 2092 is used to schedule each queue.
  • the event controller 2093 is equivalent to a state machine for event processing, because the upper layer protocol does not directly interact with the data encoding and the underlying link socket transmission, and the respective processing procedures are different, resulting in incomplete data generation and consumption time. Synchronous; coupled with external dynamic control 2094 to modify parameter events, network adaptive control 2096 reporting network events are all possible at any time, completely asynchronous; event controller 2093 has an internal event queue, which will divide events into different event processing threads and Event queue, each processing thread corresponds to an event queue; currently there are 2 processing threads and 2 event processing queues inside; sending thread and receiving thread.
  • the external dynamic control 2094 reports network events or information such as the delay packet loss rate to the scheduling control unit 210 for decision-making.
  • the scheduling control unit 210 will send quantitative transmission indicators, such as delay, code stream size and other parameters back to the external dynamic control 2094, So that the external dynamic control 2094 adjusts the length of each queue according to these parameters.
  • Statistics 2095, receiving event controller 2093 sending thread, receiving thread, and network adaptive control 2096 receiving thread reported sending data, received data, packet loss parameters, time stamps, etc. to calculate the delay, effective bandwidth, packet loss rate , Packet error rate, RTT and other network parameters. At the same time, the statistical result is reported to the scheduling control unit 210.
  • the algorithm pool 2097 provides algorithms of different models for real-time calculation and generation of quantified results.
  • the FEC forward error correction model algorithm is used to generate redundant messages
  • the flow control algorithm is used to control the size of the network output code stream
  • the network prediction model algorithm is used for various algorithms such as network behavior prediction.
  • the scheduling control unit 210 will specify the target delay and queue length of a certain queue, adjust the queue priority, and control the redundancy generation model to recover the lost data according to the error correction algorithm without retransmitting the data; synchronization control tone
  • the sending time of each video queue because audio and video need to be played synchronously controlled on the remote device 300; during the transmission process, the audio and video queues need to be dispatched as fair as possible to keep the audio and video data in general synchronization, so as to reduce the remote device 300's Deal with errors.
  • the redundant generation model includes five units, an algorithm unit (AM), a control unit (CM), a redundant calculation unit (RM), a transmission unit (TM), and a data analysis unit (PM).
  • AM algorithm unit
  • CM control unit
  • RM redundant calculation unit
  • TM transmission unit
  • PM data analysis unit
  • the algorithm unit contains various algorithms for generating redundancy models, including but not limited to convolutional codes, Hamming codes, BCH codes, RS codes, Turbo codes, LDPC codes and TPC codes (some units may require special hardware support).
  • the control unit CM is a collection of various strategies, hardware platform screening strategies, data classification strategies, network state judgment strategies and other different decision-making mechanisms. Comprehensive judgment is made according to the data fed back by each unit, and different redundancy algorithms in AM are called to finally generate a redundancy model. CM should balance fault tolerance, computing power and load ratio when choosing a model.
  • the redundancy calculation unit RM generates redundancy for the grouped data according to the redundancy model, the corresponding check code algorithm and the control subcontracting strategy, and then sends it.
  • the transmission unit TM is responsible for generating data transmission and detecting the status of the link, such as packet loss rate, error packet rate, RTT, TTL, etc. These status data are fed back to the control unit as input parameters for the next stage of transmission.
  • the data analysis unit PM analyzes the received data to determine whether there is a packet loss. If there is a packet loss, the redundant calculation method is reversed based on the existing information to recover the lost data.
  • the redundant generation model can be divided into a sender and a receiver.
  • the CM unit After the data is input into the system, the CM unit first divides the data transmission priority according to the data classification strategy, according to the important and urgent procedures of the data and the size of the data; CM, then receives the network status data fed back by TM, According to the data priority and network status, various algorithms in AM are called to establish a redundancy model and update it to RM. RM calculates the result according to the redundancy model, and then sends it to the transmission unit through the network.
  • Receiving end process After receiving the data, TM parses out the information and inputs it to CM. CM calls AM based on the analysis result to build a redundant model and updates it to PM. At this time, TM must determine whether there is packet loss or loss. The package enters the relevant data into the PM and recovers the lost data.
  • the upper layer can adopt different transmission strategies for different types of data.
  • transmission control 209 and transmission control 308 each type of data corresponds to an independent queue for transmission scheduling.
  • the queue length is different, and the queue length will affect the transmission delay.
  • the delay requirements are slightly different in different scenarios.
  • the user perception is not high.
  • the queue length can be increased.
  • the redundancy ratio (reducing computing power and increasing the effective load ratio).
  • the queue priority is the lowest; but when editing the document, the delay requirement is higher, because the user has been interacting with the source computer at this time.
  • the operation response requirements are high.
  • the queue length will be reduced, the redundancy ratio will be increased to reduce the delay, and the smoothness of user operations will be ensured.
  • the queue priority is increased to second only to the mouse and keyboard data.
  • the audio is relatively simple, divided into sound and no sound.
  • the length of the queue when there is sound can be determined according to the length of the video queue. Because audio and video synchronization is required, the queue priority can be equal to the video, because when there is no sound No data is sent, and other queue scheduling is not affected. If there is no sound, the queue length can be set to 1 (it can be slightly longer to prevent sudden sound) or not set.
  • Signaling refers to the interactive data generated in the process of establishing a connection between the source 200 and the remote 300.
  • the amount of data is small, but it is very important.
  • the delay requirement is low (second level).
  • the redundancy can be increased to 100% or even higher.
  • the queue length can be set to 10% of the total data volume. For example, if there are 100 signaling per second, the queue length is initially 10, and the queue priority is higher than video playback, but lower Video queue for text operations.
  • the frequency of keyboard and mouse events is higher, and the number of generated is an order of magnitude higher than that of audio and video, especially for the mouse, but each event has a small amount of data, requires high latency and high accuracy, and the queue length is generally higher than the video queue length An order of magnitude. For example, if the video is 20, the keyboard and mouse may be 200, and the keyboard and mouse queue has the highest priority. The length of each of the above queues will be dynamically adjusted according to the current operation scenario and delay requirements of the scheduling feedback.
  • the execution instruction comes from the scheduling control unit 210.
  • 2091 When the transmission unit receives a piece of data, 2091 will send the data to different data queues according to the category (ie, the audio, video, keyboard and mouse operation event queues mentioned above, etc.), and 2091 will send it to the event queue of the 2093 scheduling thread
  • the category ie, the audio, video, keyboard and mouse operation event queues mentioned above, etc.
  • 2091 will send it to the event queue of the 2093 scheduling thread
  • For a sending data event after the sending thread receives the event, there is only one sending thread, which may correspond to events generated by multiple upper-layer protocols. These events need to be combined and filtered, and then a scheduling event will be generated and then the transmission queue scheduling 2092 will be triggered to work; Queue scheduling 2092 performs scheduling.
  • the network adaptive control 2096 encapsulates the data (encapsulation is for Unpack, and generate redundancy and add additional control information, used for packet loss data recovery or retransmission), and send it to the data buffer area of the bottom socket of the network 2098.
  • the network 2098 has a dedicated sending thread that runs all the time, and the data in the read buffer is continuously sent out.
  • the network 2098 also has a dedicated receiving thread that runs continuously corresponding to the physical link to ensure receiving efficiency. After receiving the data, it generates a receiving event to the network adaptive control 2096.
  • the network adaptive control 2096 also has an independent receiving thread from the network 2098 buffer area. Read the data, and then decapsulate (decapsulation is to check the data to check whether there is packet loss. If there is no packet loss, remove the redundancy and control information, and then fill it to the specified position. If there is packet loss, pass the control information Perform calculations with redundancy to recover. If there is too much loss and cannot be calculated, a packet loss report must be sent to the source, and the source will retransmit the corresponding data).
  • the decapsulation After the decapsulation is completed, it will be judged whether it corresponds to a complete data message of the upper layer protocol. If a report event is generated, it is processed by the receiving thread of the event controller 2093.
  • the receiving thread of the event controller 2093 is responsible for sending the data back to the upper application through the report interface registered by the display protocol and description mapping 2091; if there is a packet loss event,
  • the network adaptive control 2096 will also report packet loss parameters, including the number of packets and size data, to the statistics thread of the event controller 2093.
  • the description of the transmission queue scheduling 2092 scheduling each queue is as follows: the transmission queue scheduling 2092 determines which queue data should be sent at the current time according to the display protocol and description mapping 2091 description and the feedback of the network adaptive control 2096, and the transmission queue scheduling 2092 is in When there is data in the queue, it will keep running until all the data in the queue is sent out.
  • the transmission queue scheduling 2092 follows the following formula during operation. That is to say, every time when the transmission queue scheduling 2092 selects which message in the transmission queue, it can find out the message that can be transmitted fastest in all queues according to the following formula, and select the message for transmission until All messages in the queue are sent.
  • virt_st transmission start time. Initially, since virt_ft does not yet exist, last(virt_ft) does not exist; virt_st is the generation time of the earliest message among all the ats messages in the queue; among them, max is It means to find out the earliest message generated in the queue, and recalculate every subsequent scheduling. Therefore, virt_st is a weight data and cannot be counted as the real transmission start time;
  • virt_ft Estimated transmission completion time. In fact, this is a weighted data, not the actual transmission completion time;
  • MAC_size The larger the message size, the longer it will take to send
  • R is the current total available bandwidth
  • Wi refers to the weight (priority designated by 210) data of each queue
  • This algorithm is an improvement on the operating system job scheduling algorithm, adding queue priority parameters to adapt to the existing transmission of different priority data.
  • Network adaptive control 2096 will establish a closed-loop prediction mechanism, based on the results of statistics 2095 and external dynamic control 2094 input, and then predict the network status of the sending cycle according to the algorithm pool 2097 network prediction model, which can predict the network bandwidth delay and some networks Events, such as congestion, large delay, abnormal packet loss, burst traffic, disorder, etc., are reported back to affect the upper-layer output code stream size and output speed or adjust the current transmission strategy by self-consumption.
  • the network adaptive control 2096 will report the current status, and the scheduling control unit 210 will control the upper layer.
  • Each coding unit increases the code stream and improves the quality to improve the output quality of the remote display device.
  • adaptive 2096 needs to control the priority transmission of control data. The amount of such data is small, the transmission quality is high, and the amount of data is large. For data with low transmission quality requirements, check whether the display protocol and description map 2091 can be discarded to reduce transmission pressure.
  • the sending buffer adjusts the sending buffer to prevent non-discardable data loss.
  • the current network bandwidth is sufficient and the delay is low, but the error packet rate is high.
  • the FEC model will be adjusted to generate more redundant packets to ensure that the data can be restored without retransmission.
  • the prediction mechanism model is shown in Figure 16 (including but not limited to the following input and output conditions).
  • the prediction mechanism model can be based on a neural network model.
  • the input parameters may include: receiving bandwidth, retransmission ratio, packet loss rate, network delay and inter-packet delay, and output parameters may include: bandwidth and delay.
  • Input the input reference to the original neural network model, and train the original neural network model.
  • the model is trained and the trained neural network is The model is used as a predictive mechanism model for practical application.
  • Application networks can be divided into local area networks and wide area networks. Since the conditions of the two networks are quite different, in order to obtain better decision-making results, the training data can be divided into two groups, so as to obtain two prediction mechanism models for the local area network and the wide area network respectively. This can effectively reduce training complexity and improve the accuracy of model prediction results.
  • the prediction mechanism model training needs to be continued in actual applications, that is, first use the existing data to train a basic model, and then test and use it, and continue to use it. Conduct training.
  • output parameters with good prediction results and corresponding input parameters can be recorded in the current network state, and used to train the prediction mechanism model so that the prediction mechanism model can be adjusted according to the current network state.
  • the scheduling control unit 210 also has a supplementary image change prediction mechanism to assist the encoding when the image processing device 200 is independently used outside the host 100.
  • the image processing device 200 collects information such as drawing instructions, image changes, and image movement that cannot be obtained from the host 100. Without this unit, it needs to compare pixel by pixel for image recognition to generate motion vectors and change positions. information.
  • the image change prediction mechanism of this unit establishes a corresponding relationship between user operations and image changes. Different user operations will affect the changes in the image, and generate these changes to assist the work of the coding unit.
  • the prediction mechanism model is shown in Figure 17.
  • the main function of the prediction mechanism unit is to assist the coding unit in image recognition, so as to minimize the coding consumption and time delay.
  • the prediction mechanism unit can be a machine unit learning algorithm model, such as neural network, k-nearest neighbor, Bayesian, decision tree, svm, logistic regression, maximum entropy model, hidden Markov, conditional random field, adaboost, em, etc.
  • An algorithm preferably a neural network. Because the larger the amount of training data, the better the neural network training results. At present, we have enough data to train the neural network model to make the prediction effect of the trained prediction mechanism model better.
  • Input parameters can include: image recognition algorithm, bandwidth, and keyboard and mouse control time.
  • Output parameters can include: movement and change area. Among them, the result of output parameters in training data can be determined based on Calculated by the image recognition algorithm of pixel comparison.
  • Input the input parameters into the original neural network model, and train the original neural network model.
  • the model is trained and the trained neural network is The model is used as a prediction mechanism model for practical application.
  • the output result of the prediction mechanism model can be sent to the coding unit.
  • the use of the prediction mechanism model to calculate the movement and change area can reduce the consumption of computing resources (it is impossible to completely eliminate , Because the forecast is only an aid, the result may not be completely correct), and reduce the amount of calculation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

La présente divulgation concerne le domaine technique du codage informatique, et porte sur un appareil de traitement d'image, un dispositif distant et un système de communication. L'appareil comprend : une unité d'acquisition d'image, une unité de codage d'image, une unité d'acquisition de son, une première unité de codage de son et une première unité de commande de transmission. L'unité d'acquisition d'image sert à acquérir une image d'ordinateur selon un premier paramètre de commande prédéfini. L'unité d'acquisition de son sert à acquérir des premières données sonores selon le paramètre. L'unité de codage d'image sert à effectuer un codage d'image sur l'image d'ordinateur selon un deuxième paramètre de commande prédéfini pour obtenir des premières données codées. La première unité de codage de son sert à effectuer un codage de son sur les premières données sonores pour obtenir des secondes données codées. La première unité de commande de transmission sert à envoyer les premières données codées et les secondes données codées au dispositif distant selon un troisième paramètre de commande prédéfini. Selon les modes de réalisation de la présente divulgation, des paramètres de transmission peuvent être ajustés de manière dynamique dans le système de communication pour transmettre des données sonores et d'image.
PCT/CN2020/130436 2020-01-09 2020-11-20 Appareil de traitement d'image, dispositif distant et système de communication WO2021139418A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010020909.9 2020-01-09
CN202010020909.9A CN111182220A (zh) 2020-01-09 2020-01-09 图像处理装置、远端设备及通信系统

Publications (1)

Publication Number Publication Date
WO2021139418A1 true WO2021139418A1 (fr) 2021-07-15

Family

ID=70657843

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/130436 WO2021139418A1 (fr) 2020-01-09 2020-11-20 Appareil de traitement d'image, dispositif distant et système de communication

Country Status (2)

Country Link
CN (1) CN111182220A (fr)
WO (1) WO2021139418A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024082971A1 (fr) * 2022-10-19 2024-04-25 腾讯科技（深圳）有限公司 Procédé de traitement vidéo et dispositif associé

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111182220A (zh) * 2020-01-09 2020-05-19 西安万像电子科技有限公司 图像处理装置、远端设备及通信系统
CN112788193A (zh) * 2020-12-30 2021-05-11 北京达佳互联信息技术有限公司 图像传输方法、装置、电子设备及存储介质
CN116530079A (zh) * 2021-01-08 2023-08-01 深圳市大疆创新科技有限公司 编码方法、解码方法和编码装置、解码装置
CN113810639B (zh) * 2021-09-28 2023-09-29 深圳万兴软件有限公司 一种录制鼠标信息可再编辑的方法、装置及相关介质
CN115225520B (zh) * 2022-07-15 2023-09-26 同济大学 一种基于元学习框架的多模态网络流量预测方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102223516A (zh) * 2010-04-14 2011-10-19 奥多比公司 连接的媒体通信设备间的媒体质量增强
CN104768026A (zh) * 2015-04-17 2015-07-08 中国工商银行股份有限公司 一种多信道音视频转码装置
US20150201198A1 (en) * 2014-01-15 2015-07-16 Avigilon Corporation Streaming multiple encodings encoded using different encoding parameters
CN105429983A (zh) * 2015-11-27 2016-03-23 刘军 采集媒体数据的方法、媒体终端及音乐教学系统
US20170094296A1 (en) * 2015-09-28 2017-03-30 Cybrook Inc. Bandwidth Adjustment For Real-time Video Transmission
CN109101209A (zh) * 2018-09-05 2018-12-28 广州维纳斯家居股份有限公司 智能屏幕共享方法、装置、智能升降桌及存储介质
CN111182220A (zh) * 2020-01-09 2020-05-19 西安万像电子科技有限公司 图像处理装置、远端设备及通信系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102223516A (zh) * 2010-04-14 2011-10-19 奥多比公司 连接的媒体通信设备间的媒体质量增强
US20150201198A1 (en) * 2014-01-15 2015-07-16 Avigilon Corporation Streaming multiple encodings encoded using different encoding parameters
CN104768026A (zh) * 2015-04-17 2015-07-08 中国工商银行股份有限公司 一种多信道音视频转码装置
US20170094296A1 (en) * 2015-09-28 2017-03-30 Cybrook Inc. Bandwidth Adjustment For Real-time Video Transmission
CN105429983A (zh) * 2015-11-27 2016-03-23 刘军 采集媒体数据的方法、媒体终端及音乐教学系统
CN109101209A (zh) * 2018-09-05 2018-12-28 广州维纳斯家居股份有限公司 智能屏幕共享方法、装置、智能升降桌及存储介质
CN111182220A (zh) * 2020-01-09 2020-05-19 西安万像电子科技有限公司 图像处理装置、远端设备及通信系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024082971A1 (fr) * 2022-10-19 2024-04-25 腾讯科技（深圳）有限公司 Procédé de traitement vidéo et dispositif associé

Also Published As

Publication number Publication date
CN111182220A (zh) 2020-05-19

Similar Documents

Publication Publication Date Title
WO2021139418A1 (fr) Appareil de traitement d'image, dispositif distant et système de communication
CN111246262A (zh) 编码调度控制方法及调度控制器
RU2497304C2 (ru) Динамическая модификация свойств видео
CN109743600B (zh) 基于可穿戴的现场运维自适应视频流传输速率控制方法
CN110430441B (zh) 一种云手机视频采集方法、系统、装置及存储介质
US20110299588A1 (en) Rate control in video communication via virtual transmission buffer
US8675728B2 (en) Transmitting apparatus and method, and receiving apparatus and method
CN106488265A (zh) 一种发送媒体流的方法和装置
CN106993190B (zh) 软硬件协同编码方法及系统
US10225516B2 (en) Latency mitigation through intelligent extrapolation in multimedia systems
CN112995636B (zh) 基于边缘计算和主动缓存的360度虚拟现实视频传输系统及参数优化方法
JPWO2007007496A1 (ja) 多地点会議システム、多地点会議方法及びプログラム
CN111010582A (zh) 一种云桌面图像处理方法、装置、设备及可读存储介质
WO2020207487A1 (fr) Procédé, appareil et système de traitement
CN103780907B (zh) 一种视频数据流量整形的方法和装置
US20110235709A1 (en) Frame dropping algorithm for fast adaptation of buffered compressed video to network condition changes
CN109874027A (zh) 一种低延时手术示教直播方法及其系统
CN110572695A (zh) 媒体数据的编码、解码方法及电子设备
CN106210443B (zh) 网络控制显示内容同步的方法
CN112866746A (zh) 一种多路串流云游戏控制方法、装置、设备及存储介质
CN112929712A (zh) 视频码率调整方法和装置
US20220255981A1 (en) Method and Apparatus for Adjusting Attribute of Video Stream
CN115243074A (zh) 视频流的处理方法及装置、存储介质、电子设备
US20220408097A1 (en) Adaptively encoding video frames using content and network analysis
CN112565670B (zh) 云会议多层视频快速平滑出图的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20912634

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20912634

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27-01-2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20912634

Country of ref document: EP

Kind code of ref document: A1