CN117793386A

CN117793386A - Data buffering method, data transmission system, device, equipment and storage medium

Info

Publication number: CN117793386A
Application number: CN202311852824.7A
Authority: CN
Inventors: 余宗璞
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2023-12-29
Filing date: 2023-12-29
Publication date: 2024-03-29

Abstract

The application discloses a data buffering method, a data transmission system, a device, equipment and a storage medium, which relate to the technical field of data transmission and are used for stably and conveniently buffering and storing data. The method comprises the following steps: receiving a multimedia data packet sent by target acquisition equipment, and determining storage resources occupied by the multimedia data packet; based on storage resources occupied by the multimedia data packets sent by the target acquisition equipment, adjusting the capacity of a buffer area, wherein the adjusted available capacity of the buffer area is larger than the storage resources occupied by the multimedia data, and the available capacity is used for representing the storage resources of the non-stored data; storing the multimedia data packet in the adjusted buffer; and sending the multimedia data packet stored in the buffer area to the first user equipment so that the first user equipment plays the multimedia resource based on the multimedia data packet.

Description

Data buffering method, data transmission system, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of data transmission technologies, and in particular, to a data buffering method, a data transmission system, a device, an apparatus, and a storage medium.

Background

Users often need to view multimedia resources (such as video and audio) acquired by the acquisition device, and due to the limitation of the performance of the acquisition device, the acquisition device only supports transmitting the multimedia resources to a small number of user devices needing to view the acquired multimedia resources at the same time, for example, most of the acquisition devices only support transmitting 6 paths of multimedia resources at the same time, and when the number of the user devices needing to view the acquired multimedia resources exceeds 6, the acquisition device cannot transmit the multimedia resources to each user device at the same time. Therefore, the multimedia resources acquired by the acquisition device need to be forwarded or distributed to a plurality of user devices by means of a server.

Currently, there is a method of modifying the size of a buffer zone based on storage resources occupied by multimedia data transmitted by a model predictive server by establishing the buffer zone in a user equipment, and buffering the multimedia data to adapt to a code stream change in a multimedia data transmission process. However, this method needs to deploy a model on each user device that needs to view the multimedia data collected by the collection device, and if the performance of the user device is insufficient or the network environment of the user device changes, the buffer area will be affected, so as to affect the playing effect of the multimedia data.

Therefore, a more stable and more convenient data buffering method is needed.

Disclosure of Invention

The application provides a data buffering method, a data transmission system, a device, equipment and a storage medium, which are used for stably and conveniently buffering data.

In order to achieve the technical purpose, the application adopts the following technical scheme:

in a first aspect, an embodiment of the present application provides a data buffering method, applied to a server, where the method includes:

receiving a multimedia data packet sent by target acquisition equipment, and determining storage resources occupied by the multimedia data packet;

based on storage resources occupied by the multimedia data packets sent by the target acquisition equipment, adjusting the capacity of a buffer area, wherein the adjusted available capacity of the buffer area is larger than the storage resources occupied by the multimedia data, and the available capacity is used for representing the storage resources of the non-stored data;

storing the multimedia data packet in the adjusted buffer;

and sending the multimedia data packet stored in the buffer area to the first user equipment so that the first user equipment plays the multimedia resource based on the multimedia data packet.

The technical scheme provided by the application at least brings the following beneficial effects: through establishing the buffer area in the server, carry out dynamic adjustment to the size of buffer area based on the multimedia data package that target acquisition equipment sent, when the user wants to look over the multimedia resource that target acquisition equipment gathered, the server can be direct with the multimedia data package transmission that stores in the buffer area for user equipment, satisfies user's the demand of looking over, need not to change user equipment, and data buffering process is more convenient, also can not influence data buffering process because of user equipment's performance variation, and data buffering process is more stable. Meanwhile, when the code rate of the multimedia data packet sent by the acquisition equipment is increased, the capacity of the buffer is correspondingly increased, more received multimedia data packets can be stored, and the adjustment requirement on the buffer area when the code stream changes in the process of buffering the multimedia data is met.

In one possible implementation, the buffer includes a plurality of buffer nodes; based on storage resources occupied by the multimedia data packets, adjusting the capacity of the buffer area includes: when the storage resource occupied by the currently received multimedia data packet is larger than the available capacity of the current buffer node for the data to be stored, a buffer node is newly added; storing the multimedia data in the adjusted buffer, comprising: and storing the currently received multimedia data packet in a current buffer node and a newly added buffer node of the data to be stored.

In one possible implementation, the method further includes: and when the multimedia data packet sent by the target acquisition equipment is received for the first time, generating a first buffer node of the buffer area according to the preset capacity.

In one possible implementation, the buffer includes a plurality of buffer nodes linked; the method further comprises the steps of: if the buffer period is determined to be ended, recording the total storage resources occupied by the buffer period; and adjusting the capacity of the buffer zone based on the total storage resources occupied by the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, the adjusting the capacity of the buffer area based on the total storage resources occupied by each of the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer area includes: determining a storage resource mean value based on the total storage resources occupied by each of the latest preset number of buffer periods; and adjusting the capacity of the buffer zone based on the storage resource mean value and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, adjusting the capacity of the buffer based on the storage resource mean and the capacity of the buffer nodes in the buffer includes: if the storage resource mean value is smaller than the total capacity of the buffer nodes except the last buffer node in the plurality of linked buffer nodes in the buffer zone, deleting the last buffer node in the buffer zone until the total capacity of the buffer zone is larger than or equal to the storage resource mean value, and the total capacity of the buffer nodes except the last buffer node in the buffer zone is smaller than the storage resource mean value; and determining the buffer areas with the total capacity larger than or equal to the average value of the storage resources and the total capacity of the buffer nodes except the last buffer node smaller than the average value of the storage resources as the buffer areas after shrinking.

In one possible implementation, before adjusting the capacity of the buffer based on the storage resources occupied by the multimedia data packets, the method further includes: analyzing the currently received multimedia data packet; if the buffer period starting identification is analyzed from the currently received multimedia data packet, determining that the current buffer period starts, ending the previous buffer period, and clearing the currently stored multimedia data packet in the buffer area.

In one possible implementation, the multimedia data packet is a video frame data packet, the buffering period is a key frame period, and the starting identifier of the buffering period is a key frame; if the buffer period starting identifier is parsed from the currently received multimedia data packet, determining that the current buffer period starts and the last buffer period ends, including: if the key frame is analyzed from the video frame data packet received currently, determining that the current key frame period starts and ending the previous key frame period.

In a second aspect, the present application provides a data transmission system comprising:

the acquisition equipment is used for generating a multimedia data packet and sending the multimedia data packet to the server;

the server is used for executing any one of the data buffering methods provided in the first aspect and sending the multimedia data packet stored in the buffer area of the server to the user equipment;

and the user equipment is used for receiving the multimedia data packet sent by the server and playing the multimedia resource based on the multimedia data packet.

In a third aspect, the present application provides a data buffering apparatus, comprising:

the receiving module is used for receiving the multimedia data packet sent by the target acquisition equipment;

The processing module is used for determining storage resources occupied by the multimedia data packet;

the processing module is also used for adjusting the capacity of the buffer area based on the storage resources occupied by the multimedia data packet, the available capacity of the adjusted buffer area is larger than the storage resources occupied by the multimedia data packet, and the available capacity is used for representing the storage resources of the non-stored data;

the processing module is also used for storing the multimedia data packet in the adjusted buffer area;

and the sending module is used for sending the multimedia data packet stored in the buffer area to the first user equipment so that the first user equipment plays the multimedia resource based on the multimedia data packet.

In one possible implementation, the buffer includes a plurality of buffer nodes; the processing module is specifically configured to add a buffer node when storage resources occupied by a currently received multimedia data packet are greater than an available capacity of a current buffer node of data to be stored; the processing module is specifically configured to store the currently received multimedia data packet in a current buffer node and a newly added buffer node of data to be stored.

In a possible implementation manner, the processing module is further configured to generate, when the multimedia data packet sent by the target acquisition device is received for the first time, a first buffer node of the buffer area according to a preset capacity.

In one possible implementation, the buffer includes a plurality of buffer nodes linked; the processing module is also used for: if the buffer period is determined to be ended, recording the total storage resources occupied by the buffer period; and adjusting the capacity of the buffer zone based on the total storage resources occupied by the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, the processing module is specifically configured to: determining a storage resource mean value based on the total storage resources occupied by each of the latest preset number of buffer periods; and adjusting the capacity of the buffer zone based on the storage resource mean value and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, the processing module is specifically configured to: if the storage resource mean value is smaller than the total capacity of the buffer nodes except the last buffer node in the plurality of linked buffer nodes in the buffer zone, deleting the last buffer node in the buffer zone until the total capacity of the buffer zone is larger than or equal to the storage resource mean value, and the total capacity of the buffer nodes except the last buffer node in the buffer zone is smaller than the storage resource mean value; and determining the buffer areas with the total capacity larger than or equal to the average value of the storage resources and the total capacity of the buffer nodes except the last buffer node smaller than the average value of the storage resources as the buffer areas after shrinking.

In one possible implementation, the processing module is further configured to: analyzing the currently received multimedia data packet; if the buffer period starting identification is analyzed from the currently received multimedia data packet, determining that the current buffer period starts, ending the previous buffer period, and clearing the currently stored multimedia data packet in the buffer area.

In one possible implementation, the multimedia data packet is a video frame data packet, the buffering period is a key frame period, and the starting identifier of the buffering period is a key frame; the processing module is specifically configured to determine that a current key frame period starts and that a previous key frame period ends if a key frame is parsed from a currently received video frame packet.

In a fourth aspect, the present application provides an electronic device, comprising: one or more processors; one or more memories; wherein the one or more memories are configured to store computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform any of the data buffering methods provided in the first aspect described above.

In a fifth aspect, the present application provides a computer-readable storage medium storing computer-executable instructions that, when executed on a computer, cause the computer to perform any one of the data buffering methods provided in the first aspect above.

In a sixth aspect, the present application provides a computer program product comprising computer instructions which, when run on a computer, cause the computer to perform any one of the data buffering methods provided in the first aspect above.

For a detailed description of the second aspect and various implementations thereof in this application, reference may be made to the detailed description of the first aspect and various implementations thereof; moreover, the advantages of the second aspect and the various implementations thereof may be referred to as analyzing the advantages of the first aspect and the various implementations thereof, and will not be described herein.

These and other aspects of the present application will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic structural diagram of a data transmission system to which a data buffering method according to an embodiment of the present application is applicable;

fig. 2 is a schematic hardware structure of a computing device according to an embodiment of the present application;

FIG. 3 is a flowchart of a data buffering method according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of an application scenario of a data buffering method according to an embodiment of the present application;

fig. 5 is a second application scenario schematic diagram of a data buffering method according to an embodiment of the present application;

Fig. 6 is a third application scenario diagram of a data buffering method according to an embodiment of the present application;

fig. 7 is a schematic diagram of an application scenario of a data buffering method according to an embodiment of the present application;

fig. 8 is a fifth application scenario diagram of a data buffering method according to an embodiment of the present application;

fig. 9 is a sixth application scenario schematic diagram of a data buffering method provided in an embodiment of the present application;

fig. 10 is a schematic diagram seventh of an application scenario of a data buffering method provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a data buffer device according to an embodiment of the present application.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion. The terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.

Users often need to view multimedia resources (such as video and audio) acquired by the acquisition device, and due to the limitation of the performance of the acquisition device, the number of the outflow channels of the acquisition device is limited, so that the multimedia resources cannot be simultaneously transmitted to a plurality of user devices needing to view the acquired multimedia resources. Therefore, the multimedia resources acquired by the acquisition device need to be forwarded or distributed to a plurality of user devices by means of a server.

Currently, there is a method of buffer-storing multimedia data by creating a buffer in a user equipment, changing the size of the buffer based on storage resources occupied by the multimedia data transmitted by a model predictive server. However, this method needs to deploy a model on each user device that needs to view the multimedia data collected by the collection device, and if the performance of the user device is insufficient or the network environment of the user device changes, the buffer area will be affected, so as to affect the playing effect of the multimedia data.

Therefore, a more stable and more convenient data buffering method is needed.

In this regard, the embodiment of the application provides a data buffering method, through establishing the buffer in the server, carry out dynamic adjustment to the size of the buffer based on the multimedia data package that target acquisition equipment sent, when the user wants to look over the multimedia resource that target acquisition equipment gathered, the server can directly transmit the multimedia data package that stores in the buffer to user equipment, satisfy user's the demand of looking over, need not to change user equipment, the data buffering process is more convenient, also can not influence the data buffering process because of user equipment's performance change, the data buffering process is more stable. Meanwhile, when the code rate of the multimedia data packet sent by the acquisition equipment is increased, the capacity of the buffer is increased, more received multimedia data packets can be stored, and the requirement of data buffering is met.

Referring to fig. 1, a data transmission system to which the data buffering method provided in the present application is applicable is shown. As shown in fig. 1, the data transmission system 1 includes: acquisition device 10, server 20, and user device 30.

Wherein the acquisition device 10 and the user device 30 respectively establish a communication connection with the server 20. It should be appreciated that the connection may be a wireless connection, such as a Bluetooth connection, a wireless fidelity (wireless fide lity, wi-Fi) connection, or the like; alternatively, the connection may be a wired connection, for example, an optical fiber connection, which is not limited thereto.

In some embodiments, the collection device 10 is configured to collect the multimedia asset and send the multimedia asset to the server 20. For example, the multimedia assets may be video, audio, and the like.

In some embodiments, the collecting device 10 encodes and encapsulates the collected multimedia resources, and sends the encoded and encapsulated multimedia resource code stream to the server 20.

In some embodiments, the server 20 is configured to establish a buffer corresponding to the acquisition device 10, and store the multimedia data sent by the acquisition device 10 in the buffer corresponding to the acquisition device 10.

In some embodiments, the server 20 is further configured to adjust the buffer capacity according to the received multimedia data packets. For example, the storage resource occupied by the multimedia packet currently received by the server 20 is 100KB, but the available capacity in the current buffer area is 80KB, and the server 20 expands the buffer area, where the available capacity of the expanded buffer area is greater than the storage resource occupied by the multimedia packet currently received by the server 20.

In some embodiments, the server 20 is configured to send the multimedia data packets stored in the buffer to the user device 30. The server 20 may process the multimedia data packet stored in the buffer and then forward the processed multimedia data packet to the ue 30, so as to meet the requirements of the user on the package format and the coding format of the multimedia resource.

In some embodiments, when transmitting the multimedia data packet stored in the buffer to the user equipment 30, the server 20 performs audio/video encapsulation format processing on the multimedia resource in the multimedia data packet, performs audio/video encoding format processing, and finally transmits the processed multimedia resource packet to the user equipment 30.

In some embodiments, the server 20 may include a processor and a memory, where the processor is configured to implement a process of adjusting a buffer based on a received multimedia data packet, the memory is configured to implement a process of establishing the buffer and storing the multimedia data packet sent by the acquisition device 10, and the processor is further configured to send the multimedia data packet currently stored in the memory to the user device 30. The processor may be a central processing unit (central processing unit, CPU), a general purpose processor network processor (network processor, NP), a digital signal processor (digital signal processing, DSP), a microprocessor, a microcontroller, a programmable logic device (programmable logic device, PLD), or any combination thereof. The processor may also be any other device having processing functions, such as a circuit, a device, or a software module, which is not limited in any way by the embodiments of the present application.

In some embodiments, the user device 30 is used to play multimedia assets. After receiving the multimedia data packet sent by the server 20, the user equipment 30 decodes the multimedia data packet, and further plays the multimedia resource.

In some embodiments, the capture device 10 may be a camera, video camera, audio recorder, etc., and the specific type of capture device 10 is not limited in this application.

In some embodiments, the server 20 may be a separate server or may be a server cluster, which is not particularly limited in this application.

In some embodiments, the user device 30 may be a mobile phone (mobile phone), a tablet computer, a notebook computer, a palm computer, a mobile internet device (mobile internet device, MID), a wearable device, a Virtual Reality (VR) device, an augmented reality (augmented reality, AR) device, or the like, which may play multimedia resources.

In some embodiments, one or more acquisition devices 10, one or more user devices 30 may be included in the data transmission system 1.

The hardware architecture of the server 20 described above includes the elements included in the computing device shown in fig. 2. The hardware configuration of the server 20 will be described below using the computing device shown in fig. 2 as an example.

As shown in fig. 2, the computing device may include a processor 401, a memory 402, a communication interface 403, and a bus 404. The processor 401, the memory 402 and the communication interface 403 may be connected by a bus 404.

Processor 401 is a control center of a computing device and may be a single processor or a collective term for a plurality of processing elements. For example, the processor 401 may be a general-purpose central processing unit (central processing unit, CPU), or may be other general-purpose processors. Wherein the general purpose processor may be a microprocessor or any conventional processor or the like.

As one example, processor 401 may include one or more CPUs, such as CPU 0 and CPU 1 shown in fig. 2.

Memory 402 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, as well as electrically erasable programmable read-only memory (EEPROM), magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In a possible implementation, the memory 402 may exist separately from the processor 401, and the memory 402 may be connected to the processor 401 through the bus 404 for storing instructions or program codes. The processor 401, when calling and executing instructions or program code stored in the memory 402, can implement the model deployment method provided in the embodiment of the present application.

In another possible implementation, the memory 402 may also be integrated with the processor 401.

A communication interface 403 for connecting the computing device with other devices via a communication network, which may be ethernet, a radio access network (radio access network, RAN), a wireless local area network (wireless local area networks, WLAN), etc. The communication interface 403 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

Bus 404 may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 2, but not only one bus or one type of bus.

It should be noted that the structure shown in fig. 2 is not limiting of the computing device, and the computing device may include more or less components than those shown in fig. 2, or may combine some components, or a different arrangement of components.

The implementation of the examples of the present application will be described in detail below with reference to the accompanying drawings.

As shown in fig. 3, an embodiment of the present application provides a data buffering method, which may be performed by the server 20, and the method includes the following steps:

s101, receiving a multimedia data packet sent by target acquisition equipment, and determining storage resources occupied by the multimedia data packet.

In some embodiments, the server generates the first buffer node of the buffer according to the preset capacity when receiving the multimedia data packet sent by the target acquisition device for the first time.

For example, assume that the preset capacity is 1M. The user inputs a connection instruction into the server to instruct the server to establish communication connection with the target acquisition equipment, after the communication connection is established, the target acquisition equipment encodes the acquired multimedia resources, the encoded multimedia data is sent to the server in the form of data packets, the server determines that the multimedia data packet sent by the target acquisition equipment is received for the first time by the multimedia data packet sent by the target acquisition equipment, the server establishes a buffer zone corresponding to the target acquisition equipment, and a buffer node with the capacity of 1M is generated, namely the first buffer node in the buffer zone.

In this way, when the multimedia data packet sent by the target acquisition device is received for the first time, the server establishes a buffer area corresponding to the target acquisition device, so that the multimedia data packet sent by the target acquisition device can be directly stored in the buffer area corresponding to the target acquisition device, when a user wants to view the multimedia resource acquired by the target acquisition device, the server can directly send the multimedia data packet stored in the buffer area corresponding to the target acquisition device to the user device, and the user can play the multimedia resource in the user device.

S102, adjusting the capacity of a buffer area based on storage resources occupied by the multimedia data packet.

The adjusted available capacity of the buffer area is larger than the storage resources occupied by the multimedia data packet, and the available capacity is used for representing the storage resources of the non-stored data.

In some embodiments, prior to step S103, the server parses the currently received multimedia data packet; if the buffer period starting identification is analyzed from the currently received multimedia data packet, the server determines that the current buffer period starts, the last buffer period ends, and the currently stored multimedia data packet in the buffer area is cleared.

In some embodiments, the multimedia data packet is a video frame data packet, the buffering period is a key frame period, and the buffering period start identifier is a key frame; the server determines that the current buffering period starts and the last buffering period ends, which can be specifically implemented as: if the key frame is parsed from the video frame data packet received currently, the server determines that the current key frame period starts and the last key frame period ends.

For example, taking the example of h.264 encoding of video frame data by the target acquisition device, the data structure encoded by the target acquisition device may include an I frame (intra frame), a P frame (predictive frame), and a B frame (bidirectional frame), and one key frame period (group of pictures, GOP) may include one I frame and a plurality of P frames and B frames as shown in fig. 4. The server stores video frame data in a key frame period, and as shown in fig. 5, the video frame data packet sent to the user equipment by the server includes only one I frame. When the server analyzes the I frame from the currently received video frame data packet, the server determines that the key frame period is finished, and after the currently stored video frame data in the buffer area is emptied, the capacity of the buffer area is adjusted based on the storage resources occupied by the currently received video frame data packet, and then the currently received video frame data packet is stored in the adjusted buffer area. Wherein, the I frame is an intra-frame coding frame and also is a key frame, which can be independently decoded without referring to other images, and the first frame is generally an I frame; the P frame is a forward predictive coding frame, and the decoding can be realized only by referring to the previous I frame or P frame; b frames are bi-predictive, interpolated encoded frames that can only be decoded with either the preceding P frame or the following P frame as a reference frame.

In practical applications, the capturing device captures video data and also captures audio data, so that the multimedia data packet sent to the server by the capturing device is usually an audio-video data packet, which includes both video data and audio data.

In this way, by analyzing the currently received multimedia data packet, if the starting identifier of the buffering period is analyzed, it is determined that the last buffering period is ended, a new buffering period is started, the currently stored multimedia data packet is emptied, and then the multimedia data packet of the new buffering period is stored, so that the user equipment obtains the latest multimedia resource acquired by the target acquisition equipment from the server, and the instantaneity of data transmission can be ensured.

And S103, storing the multimedia data packet in the adjusted buffer area.

In some embodiments, the buffer includes a plurality of buffer nodes. Step S102 may be specifically implemented as: when the storage resource occupied by the currently received multimedia data packet is larger than the available capacity of the current buffer node of the data to be stored, the server adds a buffer node in the buffer area. Step S103 may be specifically implemented as: the server stores the currently received multimedia data packet in the current buffer node and the newly added buffer node of the data to be stored.

In some embodiments, the plurality of buffer nodes are linked, the newly added buffer node is linked to a current buffer node of the data to be stored, and the current buffer node of the data to be stored is a last buffer node of the linked plurality of buffer nodes.

In some embodiments, the ratio between the capacity of the newly added buffer node and the capacity of the current buffer node is a preset ratio. For example, the capacity of the newly added buffer node is three times the capacity of the current buffer node.

In practical application, since the code stream data sent to the server by the acquisition device is changed in a double relationship, the capacity of the newly added buffer node in the server is twice that of the current buffer node.

As shown in fig. 6, the first buffer node has a capacity of 1M (1024×1024byte), the second buffer node has a capacity of 2M, the third buffer node has a capacity of 4M, and the fourth buffer node has a capacity of 8M, … …. If the available capacity of the current buffer node of the data to be stored is 1M and the storage resource occupied by the currently received multimedia data packet is 1.5M, the current buffer area cannot completely store the multimedia data in the current buffer period, the server newly generates a 2M buffer node, and links the 2M buffer node to the 1M buffer node, so that the expansion process of the buffer area is completed. And finally, storing the received multimedia data packet in a buffer area after capacity expansion, wherein the buffer area formed by all buffer nodes can meet the requirement of storing the multimedia data transmitted by the acquisition equipment at a high code rate.

Therefore, the server can newly increase the buffer nodes only when the storage resources of the buffer area are insufficient, so that the capacity expansion of the buffer area is realized, and the problem that the storage resources of the server are tense due to the fact that the buffer area occupies excessive idle storage resources is avoided.

In some embodiments, the server determines, from the available capacity of the buffer, a storage location of storage resources occupied by the currently received multimedia data packet, and stores the currently received multimedia data packet in the storage location.

In an example, in the first buffer node, the storage resource occupied by the multimedia data packet currently received by the server is a packet, the server divides the storage location of the packet that is consecutive to the storage location of the last stored multimedia data packet from the buffer node to the currently received multimedia data packet based on the storage location of the last stored multimedia data packet, and stores the multimedia data packet in the divided storage location. The server sequentially stores the received multimedia data packets in the buffer nodes according to storage resources occupied by the multimedia data packets, and as shown in fig. 7, the first buffer node sequentially stores the multimedia data packets received by the server, and each storage position correspondingly stores one multimedia data packet.

In still another example, if the storage resource occupied by the multimedia packet currently received by the server is N, as shown in fig. 8, if the available capacity M in the current buffer node a is smaller than N, the server is triggered to perform a buffer expansion operation, a buffer node B is newly added to the server, the buffer node B is linked to the buffer node a, the server determines the storage location of the available capacity M from the buffer node a, and determines the storage location of (N-M) from the buffer node B, and stores the multimedia packet in these storage locations, that is, stores the multimedia packet in the storage location An and the storage location B1, where the sum of the storage resources corresponding to the storage location An and the storage location B1 is the same as the storage resource N occupied by the multimedia packet.

In this way, in a buffer period, the multimedia data packets sent by the acquisition device are sequentially stored in the buffer area according to the sequence of receiving the multimedia data packets by the server, so that the multimedia data are stored more regularly, and the multimedia data can be conveniently and rapidly positioned based on the storage position.

In some embodiments, the buffer includes a plurality of buffer nodes linked; if the buffer period is determined to be ended, the server records the total storage resources occupied by the buffer period; and adjusting the capacity of the buffer zone based on the total storage resources occupied by the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer zone.

For example, the server may adjust the capacity of the buffer based on the total storage resources occupied by each of the latest preset number of buffer periods and the capacity of the buffer node in the buffer, which may be specifically implemented as: the server determines the average value of the storage resources based on the total storage resources occupied by the latest preset number of buffer periods; the server adjusts the capacity of the buffer based on the storage resource mean and the capacity of the buffer nodes in the buffer.

For example, if the preset number is 100, determining an average value of the total storage resources occupied by the latest 100 buffer periods recorded by the server, so as to reflect the change condition of the total storage resources of the buffer periods in the 100 buffer periods.

Thus, by recording the total storage resources occupied by each of the latest preset number of buffer periods and calculating the storage resource average value, the change condition of the total storage resources of the buffer periods in the latest preset number of buffer periods can be reflected, and the capacity of the buffer area can be dynamically adjusted based on the change condition of the total storage resources of the buffer periods in the latest preset number of buffer periods.

In another example, the server adjusts the capacity of the buffer based on the total storage resources occupied by each of the latest preset number of buffer periods and the capacity of the buffer node in the buffer, and may be specifically implemented as: the server determines the median of the storage resources based on the total storage resources occupied by the latest preset number of buffer periods respectively; the server adjusts the capacity of the buffer based on the median of the storage resources and the capacity of the buffer nodes in the buffer.

For example, if the preset number is 100, the server records the total storage resources occupied by the latest 100 buffer periods, and ranks the values of the storage resources from high to low or from low to high, so as to determine a median for reflecting the change condition of the total storage resources of the buffer periods in the 100 buffer periods.

Thus, by recording the total storage resources occupied by the latest preset number of buffer periods and calculating the median of the storage resources, the change condition of the total storage resources of the buffer periods in the latest preset number of buffer periods can be reflected, and the capacity of the buffer area can be dynamically adjusted based on the change condition of the total storage resources of the buffer periods in the latest preset number of buffer periods.

It should be understood that the server determines the target data based on the total storage resources occupied by each of the latest preset number of buffer cycles; the server adjusts the capacity of the buffer based on the target data and the capacity of the buffer nodes in the buffer. The target data is data that can reflect the change condition of the storage resources occupied by the buffer cycles in the latest preset number of buffer cycles, and is not limited to the storage resource average value and the storage resource median.

In some embodiments, the server adjusts the capacity of the buffer based on the storage resource average and the capacity of the buffer node in the buffer, which may be specifically implemented as: and if the storage resource average value is smaller than the total capacity of the buffer nodes except the last buffer node in the plurality of linked buffer nodes in the buffer zone, deleting the last buffer node in the buffer zone until the total capacity of the buffer zone is larger than or equal to the storage resource average value, and determining the buffer zone with the total capacity larger than or equal to the storage resource average value and the total capacity smaller than the storage resource average value as the buffer zone after the volume reduction.

For example, assume that in the process of transmitting video data from an acquisition device to a server, after a certain time of 8K resolution code stream is transmitted, the acquisition resolution is changed to 2K in the acquisition device. Taking GOP as an example of 2 seconds, the number of non-I frames spaced between two I frames is 50, and the number of frames in 1 second is 25. When transmitting 8K ultra-high definition stream data, as shown in fig. 9, the current buffer is already expanded to the buffer node E, and the buffer can buffer multimedia data in a complete GOP. After each I frame arrives and before the buffer is cleaned, the server records the total storage resources occupied by the multimedia data packet stored in the current buffer, records the total storage resources occupied by each of the latest 100 buffer periods, and calculates the average value of the 100 total storage resources. Assuming that the user sets a process of determining whether to shrink or not once every 3 buffer periods, if the process of determining whether to shrink or not starts after the buffer period ends, as shown in fig. 10, the server determines the buffer capacity of the buffer node except the last buffer node, that is, the total capacity of several buffer nodes except the buffer node E, assuming that the code rate at the time of stable transmission of 8K resolution code stream data is 60M/s, the code rate at the time of stable transmission of 2K resolution code stream data is 6.5M/s, the buffer node has been expanded to the buffer node F, the capacity of the buffer node is 32M, the total capacity of the buffer is 63M, and the total capacity of the 1 st, 2 nd, 3 rd and 4 th buffer nodes is 15M. Because GOP is 2 seconds, the space required for buffering the whole GOP is 6.5×2=13M, in the capacity shrinking strategy that the 8K resolution is changed into 2K resolution, two cleaning buffer node release processes are performed, namely the last buffer node and the last but one buffer node are respectively judged, when the last buffer node 32M is discarded, the residual buffer capacity is 31M and is larger than 13M, and deletion is confirmed; when deleting the last but one buffer node 16M, the remaining 15M confirms the deletion; when the buffer nodes with the size of 8M are deleted next time, the residual buffer capacity is less than 13M, and the buffer area is judged not to meet the capacity shrinking condition, so that the capacity shrinking of the buffer area is 15M, and the 2K resolution code stream data can be stored, namely, the server empties and releases the 5 th and 6 th nodes in the buffer area, namely, the buffer node E and the buffer node F in the buffer area are deleted, and the buffer area with the deleted buffer node E and the buffer node F is the buffer area after capacity shrinking, so that the capacity shrinking process of the buffer area is completed.

In some embodiments, the server adjusts the capacity of the buffer based on the storage resource average and the capacity of the buffer node in the buffer, which may be specifically implemented as: and deleting the buffer nodes except the first n buffer nodes in the buffer zone to obtain the buffer zone after shrinking capacity under the condition that the total capacity of the first n buffer nodes in the buffer zone is larger than or equal to the storage resource mean value and the total capacity of the first n-1 buffer nodes in the buffer zone is smaller than the storage resource mean value.

Illustratively, the buffer area includes a first buffer node (2M), a second buffer node (4M), a third buffer node (8M), a fourth buffer node (16M), a fifth buffer node (32M) and a sixth buffer node (64M), if the storage resource average value is 28M, the total capacity of the first 4 buffer nodes is 30M, greater than the storage resource average value, the total capacity of the first 3 buffer nodes is 14M, less than the storage resource average value, and the server deletes other buffer nodes except the first 4 buffer nodes, that is, the server deletes the fifth buffer node and the sixth buffer node.

Therefore, when the code rate of the data sent by the acquisition equipment to the server is reduced, the server can timely release redundant storage resources in the buffer area, the released storage resources can be used in the buffer areas corresponding to other acquisition equipment, namely, the capacity of the buffer areas is dynamically adjusted based on the code rate of the data sent by the acquisition equipment, and the waste of the storage resources in the server can be avoided.

In some embodiments, the process of adjusting the capacity of the buffer based on the median of the storage resources and the capacity of the buffer node in the buffer may refer to the process of adjusting the capacity of the buffer based on the mean of the storage resources and the capacity of the buffer node in the buffer by the server, which is not described herein.

And S104, transmitting the multimedia data packet stored in the buffer area to the first user equipment so that the first user equipment plays the multimedia resource based on the multimedia data packet.

In some embodiments, if the server receives a play instruction sent by the second user equipment, the play instruction is used for indicating to play the multimedia resource acquired by the target acquisition equipment; the server sends the multimedia data packet currently stored in the buffer area to the second user equipment, so that the second user equipment plays the multimedia resource based on the multimedia data packet.

For example, if two user devices send a play instruction indicating to play the multimedia resource collected by the target collection device, the server sends the multimedia data stored in the buffer area corresponding to the target collection device to the two user devices.

In some embodiments, when a plurality of playing windows are established in the same ue, the ue may send a playing instruction indicating to play the multimedia resources collected by the plurality of collecting devices to the server, and then the server sends the multimedia data packets stored in the buffers corresponding to the plurality of collecting devices to the ue, so that the multimedia resources collected by different collecting devices may be played in different playing windows of the ue.

Therefore, when the number of concurrent channels needs to be expanded, the number of concurrent channels can be directly expanded in the server, and compared with the number of concurrent channels expanded in the user equipment, the method is more convenient.

The technical solution shown in fig. 3 brings at least the following advantages: through establishing the buffer area in the server, carry out dynamic adjustment to the size of buffer area based on the multimedia data package that target acquisition equipment sent, when the user wants to look over the multimedia resource that target acquisition equipment gathered, the server can be direct with the multimedia data package transmission that stores in the buffer area for user equipment, satisfies user's the demand of looking over, need not to change user equipment, and data buffering process is more convenient, also can not influence data buffering process because of user equipment's performance variation, and data buffering process is more stable. Meanwhile, when the code rate of the multimedia data packet sent by the acquisition equipment is increased, the capacity of the buffer is increased, more received multimedia data packets can be stored, and the requirement of data buffering is met.

The foregoing description of the solution provided in the embodiments of the present application has been mainly presented in terms of a method. To achieve the above functions, it includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. The technical aim may be to use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the present application.

As shown in fig. 11, the embodiment of the present application further provides a data buffering device, which is used for the data buffering method shown in the above method embodiment. The data buffer device 500 includes: a receiving module 501, a processing module 502 and a transmitting module 503.

The receiving module 501 is configured to receive a multimedia data packet sent by a target acquisition device; a processing module 502, configured to determine storage resources occupied by the multimedia data packet; the processing module 502 is further configured to adjust a capacity of the buffer based on a storage resource occupied by the multimedia data packet, where the adjusted available capacity of the buffer is greater than the storage resource occupied by the multimedia data packet, and the available capacity is used to characterize the storage resource of the non-stored data; the processing module 502 is further configured to store the multimedia data packet in the adjusted buffer; and the sending module 503 is configured to send the multimedia data packet stored in the buffer to the first user equipment, so that the first user equipment plays the multimedia resource based on the multimedia data packet.

In one possible implementation, the buffer includes a plurality of buffer nodes; the processing module 502 is specifically configured to add a buffer node when a storage resource occupied by a currently received multimedia data packet is greater than an available capacity of a current buffer node of data to be stored; the processing module 502 is specifically configured to store the currently received multimedia data packet in a current buffer node and a newly added buffer node of the data to be stored.

In a possible implementation manner, the processing module 502 is further configured to generate, when the multimedia data packet sent by the target acquisition device is received for the first time, a first buffer node of the buffer according to a preset capacity.

In one possible implementation, the buffer includes a plurality of buffer nodes linked; the processing module 502 is further configured to: if the buffer period is determined to be ended, recording the total storage resources occupied by the buffer period; and adjusting the capacity of the buffer zone based on the total storage resources occupied by the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, the processing module 502 is specifically configured to: determining a storage resource mean value based on the total storage resources occupied by each of the latest preset number of buffer periods; and adjusting the capacity of the buffer zone based on the storage resource mean value and the capacity of the buffer nodes in the buffer zone.

In one possible implementation, the processing module 502 is specifically configured to: if the storage resource mean value is smaller than the total capacity of the buffer nodes except the last buffer node in the plurality of linked buffer nodes in the buffer zone, deleting the last buffer node in the buffer zone until the total capacity of the buffer zone is larger than or equal to the storage resource mean value, and the total capacity of the buffer nodes except the last buffer node in the buffer zone is smaller than the storage resource mean value; and determining the buffer areas with the total capacity larger than or equal to the average value of the storage resources and the total capacity of the buffer nodes except the last buffer node smaller than the average value of the storage resources as the buffer areas after shrinking.

In one possible implementation, the processing module 502 is further configured to: analyzing the currently received multimedia data packet; if the buffer period starting identification is analyzed from the currently received multimedia data packet, determining that the current buffer period starts, ending the previous buffer period, and clearing the currently stored multimedia data packet in the buffer area.

In one possible implementation, the multimedia data packet is a video frame data packet, the buffering period is a key frame period, and the starting identifier of the buffering period is a key frame; the processing module 502 is specifically configured to determine that the current key frame period starts and that the previous key frame period ends if a key frame is parsed from the currently received video frame data packet.

It should be noted that the division of the modules in fig. 11 is illustrative, and is merely a logic function division, and another division manner may be implemented in practice. For example, two or more functions may also be integrated in one processing module. The integrated modules may be implemented in hardware or in software functional modules.

Another embodiment of the present application also provides an electronic device, including a memory and a processor; the memory is coupled to the processor; the memory is used to store computer program code, which includes computer instructions. The processor, when executing the computer instructions, causes the electronic device to execute the steps executed by the server in the method flow shown in the method embodiment.

In actual implementation, the receiving module 501, the processing module 502 and the transmitting module 503 may be implemented by a processor of the electronic device invoking computer program code in a memory. For specific implementation, reference may be made to the description of the data buffering method section above, and details are not repeated here.

Another embodiment of the present application further provides a computer readable storage medium, where computer instructions are stored, where the computer instructions, when executed on a computer, cause the computer to perform the steps performed by a server in a method flow shown in the foregoing method embodiment.

In another embodiment of the present application, there is also provided a computer program product including computer instructions which, when executed on a computer, cause the computer to perform the steps performed by the server in the method flow shown in the method embodiment described above.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer-executable instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, a website, computer, server, or data center via a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices including one or more servers, data centers, etc. that can be integrated with the media. Usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tape), optical media (e.g., DVD), etc.

The foregoing is merely a specific embodiment of the present application. Variations and alternatives will occur to those skilled in the art from the detailed description provided herein and are intended to be included within the scope of the present application.

Claims

1. A data buffering method, applied to a server, the method comprising:

based on the storage resources occupied by the multimedia data packet, adjusting the capacity of a buffer zone, wherein the adjusted available capacity of the buffer zone is larger than the storage resources occupied by the multimedia data packet, and the available capacity is used for representing the storage resources of the non-stored data;

storing the multimedia data packet in an adjusted buffer;

and sending the multimedia data packet stored in the buffer area to first user equipment so that the first user equipment plays the multimedia resource based on the multimedia data packet.

2. The method of claim 1, wherein the buffer comprises a plurality of buffer nodes;

the adjusting the capacity of the buffer area based on the storage resources occupied by the multimedia data packet comprises:

When the storage resource occupied by the currently received multimedia data packet is larger than the available capacity of the current buffer node of the data to be stored, a buffer node is newly added;

the storing the multimedia data in the adjusted buffer comprises:

and storing the currently received multimedia data packet in a current buffer node of the data to be stored and the newly added buffer node.

3. The method according to claim 1, wherein the method further comprises:

and when the multimedia data packet sent by the target acquisition equipment is received for the first time, generating a first buffer node of the buffer area according to a preset capacity.

4. The method of claim 1, wherein the buffer comprises a plurality of buffer nodes linked;

the method further comprises the steps of:

if the buffer period is determined to be ended, recording the total storage resources occupied by the buffer period;

and adjusting the capacity of the buffer zone based on the total storage resources occupied by the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer zone.

5. The method of claim 4, wherein the adjusting the capacity of the buffer based on the total memory resources occupied by each of the latest preset number of buffer cycles and the capacity of the buffer nodes in the buffer comprises:

Determining a storage resource mean value based on the total storage resources occupied by the latest preset number of buffer periods respectively;

and adjusting the capacity of the buffer zone based on the storage resource mean value and the capacity of the buffer nodes in the buffer zone.

6. The method of claim 5, wherein adjusting the capacity of the buffer based on the storage resource mean and the capacity of the buffer nodes in the buffer comprises:

if the storage resource average value is smaller than the total capacity of the buffer nodes except the last buffer node in the plurality of linked buffer nodes in the buffer zone, deleting the last buffer node in the buffer zone until the total capacity of the buffer zone is larger than or equal to the storage resource average value, and the total capacity of the buffer nodes except the last buffer node in the buffer zone is smaller than the storage resource average value;

and determining a buffer area with the total capacity larger than or equal to the storage resource mean value and the total capacity of the buffer nodes except the last buffer node smaller than the storage resource mean value as a buffer area after capacity reduction.

7. The method according to any of claims 1-6, wherein prior to said adjusting the capacity of the buffer based on the storage resources occupied by the multimedia data packets, the method further comprises:

Analyzing the currently received multimedia data packet;

if the buffer period starting identification is analyzed from the currently received multimedia data packet, determining that the current buffer period starts, ending the previous buffer period, and clearing the currently stored multimedia data packet in the buffer area.

8. The method of claim 7, wherein the multimedia data packet is a video frame data packet, the buffering period is a key frame period, and the buffering period start is identified as a key frame;

if the buffer period start identifier is parsed from the currently received multimedia data packet, determining that the current buffer period starts and the previous buffer period ends, including:

if the key frame is analyzed from the video frame data packet received currently, determining that the current key frame period starts and ending the previous key frame period.

9. A data transmission system, comprising:

the server is configured to perform the data buffering method of any one of claims 1 to 8, and send the multimedia data packet stored in the buffer of the server to the user equipment;

The user equipment is used for receiving the multimedia data packet sent by the server and playing the multimedia resource based on the multimedia data packet.

10. A data buffering device, comprising:

the processing module is further configured to adjust a capacity of a buffer area based on a storage resource occupied by the multimedia data packet, where an available capacity of the adjusted buffer area is greater than the storage resource occupied by the multimedia data packet, and the available capacity is used to characterize a storage resource of non-stored data;

the processing module is further configured to store the multimedia data packet in an adjusted buffer area;

11. An electronic device, comprising:

one or more processors;

one or more memories;

wherein the one or more memories are configured to store computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform the data buffering method of any of claims 1-8.

12. A computer readable storage medium storing computer executable instructions which, when run on a computer, cause the computer to perform the data buffering method of any one of claims 1 to 8.

13. A computer program product comprising computer instructions which, when run on a computer, cause the computer to perform the data buffering method of any one of claims 1 to 8.