CN117596426A

CN117596426A - Low code rate transmission method, server, system and storage medium

Info

Publication number: CN117596426A
Application number: CN202311620707.8A
Authority: CN
Inventors: 石文仙
Original assignee: Xian Wanxiang Electronics Technology Co Ltd
Current assignee: Xian Wanxiang Electronics Technology Co Ltd
Priority date: 2023-11-30
Filing date: 2023-11-30
Publication date: 2024-02-23

Abstract

The embodiment of the disclosure relates to a low code rate transmission method, a server, a system and a storage medium. The method comprises the following steps: the method comprises the steps that an encoding end of a server obtains current network bandwidth data; acquiring data of preset parameters of a display end; according to the obtained data of the preset parameters of the display end and the coding algorithm, budgeting the code stream of the image video to be transmitted to obtain budgeted code flow; comparing the budget code flow with the current network bandwidth data, and determining whether to perform resolution scaling on the image video according to a comparison result; and carrying out coding transmission according to the resolution of the scaled display end. According to the embodiment of the disclosure, according to the data of the preset parameters of the display end and the coding algorithm, the code stream of the image video to be transmitted is budgeted to obtain the budgeted code flow, the budgeted code flow and the bandwidth are compared, whether the image video is scaled or not is determined according to the comparison result, the excessive exceeding of the bandwidth transmission capability of the image video code flow is avoided, and network congestion is further avoided.

Description

Low code rate transmission method, server, system and storage medium

Technical Field

The embodiment of the disclosure relates to the technical field of image transmission, in particular to a low-code rate transmission method, a server, a system and a storage medium.

Background

With the progress of technology, the terminal side will develop towards lighter and lighter weight, and the configuration and structure of the terminal are simpler and smaller.

At present, various zero-terminal solutions have been proposed, so-called zero terminals, that is terminals only need to have a screen, a simple CPU and memory, an image receiving and decoding module for processing images, and a reverse control processing module. This solution allows to greatly simplify the size and configuration of the mobile terminal. All programs of the zero terminal are operated in the cloud server, the zero terminal uses corresponding application programs by accessing the cloud server and controls the corresponding application programs by reverse control, and the server sends real-time processing pictures to the zero terminal, so that the zero terminal operates as if the zero terminal is locally.

In practical use, especially when a zero terminal with higher resolution is connected, for example, a 4K screen, the coding speed, the transmission speed and the pressure of network bandwidth are relatively high, which is easy to cause network congestion and poor transmission quality.

Accordingly, there is a need to improve one or more problems in the related art as described above.

It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

An object of embodiments of the present disclosure is to provide a low code rate transmission method, a server, a system, and a storage medium, which overcome, at least in part, one or more of the problems due to the limitations and disadvantages of the related art.

According to a first aspect of embodiments of the present disclosure, there is provided a low code rate transmission method, the method including the steps of:

the method comprises the steps that an encoding end of a server obtains current network bandwidth data;

acquiring data of preset parameters of a display end, wherein the preset parameters comprise resolution, frame rate and quantization parameters of the display end;

according to the obtained data of the preset parameters of the display end and the coding algorithm, budgeting the code stream of the image video to be transmitted to obtain budgeted code flow;

comparing the budget code flow with the current network bandwidth data, and determining whether to perform resolution scaling on the image video according to a comparison result;

and encoding the image video according to the resolution of the scaled display end and the encoding algorithm and transmitting the image video.

In an exemplary embodiment of the disclosure, the current network bandwidth data is obtained by bandwidth prediction by a network layer connected to the display terminal.

In an exemplary embodiment of the disclosure, the step of comparing the budget code traffic with the current network bandwidth data, and determining whether to perform resolution scaling on the image video according to a comparison result includes:

solving the ratio of the budget code flow to the current network bandwidth data;

if the ratio is smaller than or equal to a preset threshold value, the resolution of the image video is not scaled;

and if the ratio is greater than a preset threshold value, scaling the resolution of the image video.

In an exemplary embodiment of the present disclosure, the preset threshold is 1.

In an exemplary embodiment of the present disclosure, the scaling factor n of the resolution is calculated according to the following formula:

wherein b is the ratio of the budget code flow to the current network bandwidth data.

According to a second aspect of the embodiments of the present disclosure, there is provided a low code rate transmission server, including: the coding end is used for coding the image video by the method.

According to a third aspect of embodiments of the present disclosure, there is provided a low code rate transmission system, including:

the zero terminal comprises a decoding end, wherein the decoding end is used for receiving and decoding the image video sent by the encoding end, and performing super-resolution reconstruction on the image video when the decoding end is in a low code rate mode;

a server; and

the network layer is used for connecting the zero terminal and the server;

the server is the low code rate transmission server.

In an exemplary embodiment of the disclosure, the step of performing super-resolution reconstruction on the image video includes:

performing wavelet denoising on the decoded image;

and performing super-resolution reconstruction on the image by using a deep learning-based method SRCNN, namely performing image feature extraction, low-latitude-to-high-dimensional vector mapping, training SR according to scaling multiple n, performing image reconstruction on the amplified image by using a neural network, finally amplifying the image, and recovering detail information of the image.

In an exemplary embodiment of the disclosure, the decoding end is configured to receive the image video sent by the encoding end and decode the image video, and generate an image frame sequence in YUV format when the decoding end is not in a low code rate mode.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the low code rate transmission method described in any one of the embodiments above.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

in the embodiment of the disclosure, the code stream of the image video to be transmitted is budgeted according to the data of the preset parameters of the display end and the coding algorithm to obtain the budgeted code flow, then the budgeted code flow and the bandwidth are compared, whether the image video is scaled or not is determined according to the comparison result, the situation that the image video code flow is too large to exceed the bandwidth transmission capability is avoided, and further adverse effects caused by data transmission blocking due to network congestion are avoided.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort.

Fig. 1 illustrates a flowchart of a low code rate transmission method in an exemplary embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of comparing the budget code traffic with the current network bandwidth data, determining whether to resolution scale the image video based on the comparison result in an exemplary embodiment of the present disclosure;

fig. 3 illustrates a schematic diagram of a low code rate transmission system in an exemplary embodiment of the present disclosure;

fig. 4 is a schematic structural diagram showing functional modules of an encoding end and a decoding end in an exemplary embodiment of the present disclosure;

fig. 5 illustrates a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure;

fig. 6 illustrates a schematic structure of a program product for implementing a low code rate transmission method in an exemplary embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

In this exemplary embodiment, a low code rate transmission method is provided first, and referring to fig. 1, the method may include the following steps:

step S101, the encoding end of the server obtains the current network bandwidth data. The network bandwidth refers to the bit stream size transmitted by the network per second, and the value of the current network bandwidth data can be represented by net_bit. The current network bandwidth data is obtained through bandwidth prediction by a network layer connected with the display terminal.

Step S102, data of preset parameters of a display end are obtained, wherein the preset parameters comprise resolution, frame rate and quantization parameters of the display end. Typically the frame rate of the display is configured by default by the user or zero terminal.

Step S103, budgeting the code stream of the image video to be transmitted according to the obtained data of the preset parameters of the display end and the coding algorithm, and obtaining the budgeted code flow. The method comprises the steps of budgeting a code stream of an image video according to resolution, frame rate, quantization parameters and an encoding algorithm to obtain the size of the code stream, wherein the size of the code stream can be represented by stream_bit.

And step S104, comparing the budget code flow with the current network bandwidth data, and determining whether to perform resolution scaling on the image video according to a comparison result. For example, it may be configured to perform resolution scaling on the image video when the budget code traffic is greater than a preset value of the current network bandwidth data, where the preset value may be set manually.

Step S105, encoding and transmitting the image video according to the scaled resolution of the display end and the encoding algorithm.

According to the method, the code stream of the image video to be transmitted is budgeted according to the data of the preset parameters of the display end and the coding algorithm, the budgeted code flow is obtained, the budgeted code flow is compared with the bandwidth, whether the image video is scaled or not is determined according to the comparison result, the situation that the image video code flow is too large to exceed the bandwidth transmission capacity is avoided, and further adverse effects caused by data transmission blocking due to network congestion are avoided.

In some embodiments, referring to fig. 2, the step S104 may include the following steps:

step S201, solving a ratio of the budget code traffic to the current network bandwidth data, where the ratio is denoted by b, and b=stream_bit/net_bit.

Step S202, if the ratio is smaller than or equal to a preset threshold, the resolution of the image video is not scaled. For example, the preset threshold may be 1, but is not limited thereto, and may be 1.1, 1.2, or the like, for example.

Step S203, if the ratio is greater than the preset threshold, scaling the resolution of the image video. If b <1, not performing resolution scaling; if b >1, performing resolution scaling. Specifically, the scaling factor n is calculated according to the following formula:

the scaled resolution size is:

Width＝Worg/n

Height＝Horg/n。

both Width and Height of the resolution scale accordingly. When both Width and Height of resolution are reduced by 2 times, the code stream becomes 1/4 of the original code stream.

Next, in this exemplary embodiment, there is provided a low code rate transmission server, the server 103 including: an encoding end configured to encode the image video using the method according to any one of the foregoing embodiments.

Through the server, the code stream of the image video to be transmitted is budgeted according to the data of the preset parameters of the display end and the coding algorithm, the budgeted code flow is obtained, then the budgeted code flow is compared with the bandwidth, whether the image video is scaled or not is determined according to the comparison result, the situation that the image video code flow is too large to exceed the bandwidth transmission capacity is avoided, and further adverse effects caused by data transmission blocking due to network congestion are avoided.

The application also provides a low code rate transmission system, please refer to fig. 3, which includes a zero terminal 101, a network layer 102 and a server 103. Wherein the server 103 employs the low code rate transmission server of any of the preceding embodiments. The network layer 102 is configured to connect the zero terminal 101 and the server 103. The zero terminal 101 may be a receiving device such as a computer, a mobile phone, or a PAD, and is configured to provide an interface for displaying an image. The zero terminal 101 reports the scaled resolution to the server, and the server encodes according to the received scaled resolution, and stores the scaling multiple n in the local terminal for use by the decoder.

Referring to fig. 4, the following describes the functions of the functional modules included in the encoding side and the decoding side.

The encoding end includes: an image acquisition module 201, an encoding module 202 and a code stream transmission module 203. The decoding end comprises: a bitstream receiving module 204, a decoding module 205, and a rendering and display module 206.

Specifically, the image acquisition module 201 is configured to acquire real-time images, and send the real-time images to the encoder for encoding.

The encoding module 202 is configured to encode the received image frame sequence and generate an encoded code stream.

The code stream transmitting module 203 is configured to transmit the encoded code stream to the zero terminal 101.

The code stream receiving module 204 is configured to receive and buffer a video code stream pushed by the server 103 as a decoded data source.

The decoding module 205 is configured to decode the video code stream.

The rendering and display module 206 is used to render and display the decoded image frames.

It should be noted that, on the basis of the above embodiment, the step of performing super-resolution reconstruction on the image video includes the following steps:

step S301, wavelet denoising is carried out on the decoded image;

in step S302, super-resolution reconstruction is performed on the image by using the deep learning-based method srcan, that is, the image feature extraction, the low latitude to high dimension vector mapping, the SR training according to the scaling multiple n, and then the image reconstruction is performed on the amplified image by using the neural network, and finally the amplified image is performed, and meanwhile, the detail information of the image is recovered.

In addition, the decoding end is configured to receive the image video sent by the encoding end and decode the image video, and generate an image frame sequence in YUV format when the decoding end is not in the low code rate mode, but the decoding end is not limited thereto, and may also generate RGB format, for example.

It should be noted that although several modules of the system for action execution are mentioned in the detailed description above, this partitioning is not mandatory. Indeed, the features and functions of two or more modules described above may be embodied in one module in accordance with embodiments of the present invention. Conversely, the features and functions of one module described above may be further divided into a plurality of modules to be embodied. The components shown as modules may or may not be physical units, may be located in one place, or may be distributed across multiple network elements. Some or all modules can be selected according to actual needs to realize the purpose of the wood invention scheme. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

Referring to fig. 5, an embodiment of the present invention also provides an electronic device 300, the electronic device 300 comprising at least one memory 310, at least one processor 320, and a bus 330 connecting the different platform systems.

Memory 310 may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 211 and/or cache memory 312, and may further include Read Only Memory (ROM) 313.

The memory 310 further stores a computer program, where the computer program may be executed by the processor 320, so that the processor 320 executes the steps of the low code rate transmission method in any embodiment of the present invention, and a specific implementation manner of the computer program is consistent with the implementation manner and the achieved technical effect described in the embodiment of the low code rate transmission method, and some contents are not repeated.

Memory 310 may also include utility 314 having at least one program module 315, such program modules 315 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.

Accordingly, processor 320 may execute the computer programs described above, as well as may execute utility 314.

Bus 330 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

The electronic device 300 may also communicate with one or more external devices 340, such as a keyboard, pointing device, bluetooth device, etc., as well as with one or more devices capable of interacting with the electronic device 300, and/or with any device (e.g., router, modem, etc.) that enables the electronic device 300 to communicate with one or more other computing devices. Such communication may occur through input-output interface 350. Also, electronic device 300 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 360. The network adapter 360 may communicate with other modules of the electronic device 300 via the bus 330. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 300, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, data backup storage platforms, and the like.

The embodiment of the invention also provides a computer readable storage medium, which is used for storing a computer program, the specific implementation manner of the computer program is consistent with the implementation manner and the achieved technical effect of the embodiment of the low code rate transmission method, and part of contents are not repeated.

Fig. 6 shows a program product 400 provided in this embodiment for implementing the low code rate transmission method described above, which may employ a portable compact disc read only memory (CD-ROM) and comprise program code, and may be run on a terminal device, such as a personal computer. However, the program product 400 of the present invention is not limited thereto, and in the present invention, the readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Program product 400 may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the C programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.

Claims

1. A low code rate transmission method, the method comprising the steps of:

2. The low code rate transmission method according to claim 1, wherein the current network bandwidth data is obtained by bandwidth prediction by a network layer connected to the display terminal.

3. The low code rate transmission method according to claim 1, wherein the step of comparing the budget code traffic with the current network bandwidth data and determining whether to perform resolution scaling on the image video according to the comparison result comprises:

4. The low code rate transmission method of claim 3, wherein the preset threshold is 1.

5. The low code rate transmission method of claim 4, wherein the scaling factor n of the resolution is calculated according to the following formula:

6. A low code rate transmission server, comprising: an encoding end for encoding the image video using the method of any one of claims 1-5.

7. A low code rate transmission system, comprising:

a server; and

the network layer is used for connecting the zero terminal and the server;

wherein the server is the low code rate transmission server of claim 6.

8. The low bit rate transmission system of claim 7, wherein the step of super-resolution reconstructing the image video comprises:

performing wavelet denoising on the decoded image;

9. The low-rate transmission system according to claim 7, wherein the decoding end is configured to receive and decode the image video sent by the encoding end, and generate a sequence of image frames in YUV format when the decoding end is not in the low-rate mode.

10. A computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor realizes the steps of the low code rate transmission method according to any of claims 1 to 5.