CN116521385A - Data stream processing method, device, computer equipment and storage medium - Google Patents

Data stream processing method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN116521385A
CN116521385A CN202210079568.1A CN202210079568A CN116521385A CN 116521385 A CN116521385 A CN 116521385A CN 202210079568 A CN202210079568 A CN 202210079568A CN 116521385 A CN116521385 A CN 116521385A
Authority
CN
China
Prior art keywords
data stream
information
storage
shared memory
target data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210079568.1A
Other languages
Chinese (zh)
Inventor
严国涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210079568.1A priority Critical patent/CN116521385A/en
Publication of CN116521385A publication Critical patent/CN116521385A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/541Client-server
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/548Queue
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to a data stream processing method, apparatus, computer device, storage medium and computer program product. The method can be applied to a scene of multimedia data distribution, and comprises the following steps: acquiring a target data stream to be distributed through a multiplexing process; writing the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtaining storage information of the target data stream; writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process; reading storage information from the data stream information area through at least one distribution process, and obtaining a target data stream from the data stream information area according to the read storage information; and transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process. The method can improve the processing efficiency of the data stream.

Description

Data stream processing method, device, computer equipment and storage medium
Technical Field
The present application relates to the field of computer technology, and in particular, to a data stream processing method, apparatus, computer device, storage medium, and computer program product.
Background
With the development of computer technology, communication based on the internet has been widely used in various industries, such as personal social contact, web conference, and the like. When a plurality of users communicate through respective terminals in the communication network, the terminals of the users need to pull data streams, such as audio and video data streams, in the communication network, so that communication is realized based on the communication network. For each different terminal accessing the same network, the same data stream needs to be pulled from the network, and the processing efficiency of the data stream is low.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data stream processing method, apparatus, computer device, computer readable storage medium, and computer program product that can improve the data stream processing efficiency.
In a first aspect, the present application provides a data stream processing method. The method comprises the following steps:
acquiring a target data stream to be distributed through a multiplexing process;
writing the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtaining storage information of the target data stream;
writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process;
Reading storage information from the data stream information area through at least one distribution process, and obtaining a target data stream from the data stream information area according to the read storage information;
and transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
In a second aspect, the present application further provides a data stream processing apparatus. The device comprises:
the data stream acquisition module is used for acquiring a target data stream to be distributed through a multiplexing process;
the data stream writing module is used for writing the target data stream into a data stream storage area of the shared memory through the multiplexing process and obtaining the storage information of the target data stream;
the storage information writing module is used for writing the storage information into a data stream information area of the shared memory through the multiplexing process and sending the target data stream to a terminal associated with the multiplexing process;
the data stream reading module is used for reading the storage information from the data stream information area through at least one distribution process and obtaining a target data stream from the data stream storage area according to the read storage information;
and the data stream distribution module is used for transmitting the target data stream obtained from the shared memory to the terminal associated with the distribution process through at least one distribution process.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring a target data stream to be distributed through a multiplexing process;
writing the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtaining storage information of the target data stream;
writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process;
reading storage information from the data stream information area through at least one distribution process, and obtaining a target data stream from the data stream information area according to the read storage information;
and transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring a target data stream to be distributed through a multiplexing process;
Writing the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtaining storage information of the target data stream;
writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process;
reading storage information from the data stream information area through at least one distribution process, and obtaining a target data stream from the data stream information area according to the read storage information;
and transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
acquiring a target data stream to be distributed through a multiplexing process;
writing the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtaining storage information of the target data stream;
writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process;
Reading storage information from the data stream information area through at least one distribution process, and obtaining a target data stream from the data stream information area according to the read storage information;
and transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
The data stream processing method, the device, the computer equipment, the storage medium and the computer program product are characterized in that the obtained target data stream to be distributed is written into a data stream storage area of a shared memory through a multiplexing process, the storage information of the obtained target data stream is written into a data stream information area of the shared memory, the target data stream is sent to a terminal associated with the multiplexing process, the storage information is read from the data stream information area through at least one distribution process, the target data stream is obtained from the data stream storage area according to the read storage information, and the target data stream obtained from the shared memory is sent to the terminal associated with the distribution process through the distribution process. In the data stream processing process, the obtained target data stream is written into a data stream storage area of the shared memory through a multiplexing process associated with the terminal, and storage information of the target data stream is written into a data stream information area of the shared memory, so that the target data stream stored in the shared memory can be quickly read and sent to the associated terminal through a distribution process, when a plurality of terminals need to obtain the same data stream, the number of processes for directly obtaining the data stream is reduced, the data stream is directly read from the shared memory through the distribution process for distribution, and the processing efficiency of the data stream is improved.
Drawings
FIG. 1 is an application environment diagram of a data stream processing method in one embodiment;
FIG. 2 is a flow diagram of a method of processing a data stream in one embodiment;
FIG. 3 is a schematic diagram of a terminal in one embodiment communicating via a server connection data flow network;
FIG. 4 is a flow chart of a read target data stream according to one embodiment;
FIG. 5 is a schematic diagram illustrating a process of accessing a media network by a first terminal according to one embodiment;
FIG. 6 is a schematic diagram illustrating a process of accessing a media network by a second terminal according to one embodiment;
FIG. 7 is a schematic diagram illustrating a process of accessing a third terminal to a media network according to an embodiment;
FIG. 8 is a system architecture block diagram of a video conferencing system in one embodiment;
FIG. 9 is a flow chart illustrating a process of writing to a shared memory in one embodiment;
FIG. 10 is a flow chart illustrating a process of reading shared memory according to one embodiment;
FIG. 11 is a block diagram of a data stream processing apparatus in one embodiment;
fig. 12 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The data stream processing method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein a plurality of terminals 102 communicate with a server 104 via a communication network, respectively. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The server 104 writes the obtained target data stream to be distributed into a data stream storage area of a shared memory through a multiplexing process, the shared memory can be preset in a data storage system, the server 104 writes the obtained storage information of the target data stream into a data stream information area of the shared memory, and the target data stream is sent to a terminal associated with the multiplexing process; further, the server 104 reads the storage information from the data stream information area through at least one distribution process, obtains the target data stream from the data stream storage area according to the read storage information, and sends the target data stream obtained from the shared memory to the terminal associated with the distribution process through the distribution process.
The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be intelligent voice interaction devices, intelligent home appliances, vehicle terminals, aircrafts and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, and the like. The terminal 102 and the server 104 may be directly or indirectly connected through wired or wireless communication, which is not limited herein. Further, cloud technology (Cloud technology) refers to a hosting technology that unifies serial resources such as hardware, software, network, etc. in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business mode, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.
In one embodiment, as shown in fig. 2, a data stream processing method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
step 202, obtaining a target data stream to be distributed through a multiplexing process.
The data stream is data that the server needs to distribute to each connected terminal to realize communication, and specifically may include various media data streams such as an audio data stream, a video data stream, an image data stream, a text data stream, and the like, where a specific data form and a data content of the data stream are determined correspondingly according to an actual application scenario. For example, in a network conference scenario, the data stream is audio/video data that needs to be distributed to each terminal; in a network live scenario, the data stream may be live media data that needs to be distributed to the audience terminals in the live room. The target data stream is a data stream to be subjected to distribution processing, and specifically may be an audio data stream to be distributed to each terminal by the server. According to different practical application scenes, the target data stream can be a real-time data stream or a periodically acquired data stream.
A process is a running activity of a program with a certain independent function in a computer with respect to a certain data set, and a process is an entity of the program and is an instance of the running program. For example, when running an instant messaging application, the computer may initiate a corresponding process for the instant messaging application. When a computer runs a plurality of programs, a plurality of processes are correspondingly run. For the server, the process may be associated with a connected terminal, that is, for a terminal connected with the server, at least one process associated with the terminal may be started to implement data stream processing of the terminal, which specifically includes forwarding an uplink data stream sent by the terminal, distributing a downlink data stream in a data stream network to the terminal, and so on. For example, when a plurality of terminals participating in the network conference are accessed in the network conference, the server may start a plurality of processes to realize audio and video data transmission processing of each terminal. The multiplexing process is a designated process for realizing multiplexing, that is, the multiplexing process can realize a plurality of functions, specifically including acquisition and distribution processing of the data stream, and multiplexing processing of the data stream. In the processing process of the data stream, the server can be connected with a plurality of terminals, and each terminal starts an associated process and realizes the communication of the terminal through the process associated with the terminal.
For example, the server is connected to the terminal 1, the terminal 2, the terminal 3, and the terminal 4, respectively, and each terminal connected to the server may access to a different data stream network for communication, or may access to the same data stream network for communication. The specific terminal 1 and the terminal 2 can be accessed into the data stream network A, and the terminal 3 and the terminal 4 can be accessed into the data stream network B, so that the terminal 1 and the terminal 2 can communicate based on the data stream acquired in the data stream network A, and the terminal 3 and the terminal 4 can communicate based on the data stream acquired in the data stream network B. Specifically, in the communication process, for the terminal 1 and the terminal 2, the server may acquire data streams from the data stream network a through processes corresponding to the terminal 1 and the terminal 2, and distribute the acquired data streams to the terminal 1 and the terminal 2 respectively; likewise, the server obtains the data stream from the data stream network B through the processes corresponding to the terminal 3 and the terminal 4, and distributes the obtained data stream to the terminal 3 and the terminal 4, respectively, where the terminal 1 and the terminal 2 may obtain the same data stream, and the terminal 3 and the terminal 4 may obtain the same data stream. The multiplexing process can realize multiplexing of the pulled data streams besides pulling the data streams to be distributed from the data stream network, so that the processes associated with other terminals in the same data stream network can directly distribute the data streams pulled by the multiplexing process, and the pulling processing from the data stream network is avoided. For example, for the data stream network a, the terminal 1 and the terminal 2 are accessed, the server starts the process 1 associated with the terminal 1 and the process 2 associated with the terminal 2, if the server designates the process 1 as a multiplexing process, the process 1 pulls the target data stream from the data stream network a, distributes the target data stream to the terminal 1, and multiplexes the target data stream based on the process 1, so that the process 2 can directly distribute the target data stream pulled by the process 1 to the terminal 2, thereby avoiding the process 2 from repeatedly pulling the target data stream from the data stream network a.
The multiplexing process may be selected from processes associated with each terminal connected to the server, for example, may be determined according to a start sequence of the processes associated with each terminal, and a process started first is designated as a multiplexing process to obtain a target data stream to be distributed. The multiplexing process may be selected from the processes associated with the terminals according to the busyness of the processes associated with the terminals connected to the server, for example, a process with the lowest busyness of the processes associated with the terminals may be selected as the multiplexing process. In addition, the multiplexing process may be a process newly created independently of a process associated with each terminal, so that the multiplexing process may acquire the target data stream to be distributed.
Specifically, when the server performs data stream processing, the server determines a multiplexing process, and specifically, the multiplexing process may be selected from processes associated with terminals connected to the server, for example, a process associated with a terminal connected to the server first may be selected as a multiplexing process, and a target data stream to be distributed is acquired through the multiplexing process. If the target data stream to be distributed to each terminal can be pulled from the data stream network through the multiplexing process, the data stream network can be a transmission network of each terminal when communication is carried out.
And 204, writing the target data stream into a data stream storage area of the shared memory through the multiplexing process, and obtaining storage information of the target data stream.
The shared memory is the same section of physical memory shared among different processes, namely, a plurality of processes can access the same memory space. Processes may connect the same piece of physical memory to their own address space and all processes may access addresses in shared memory. If a process writes data to shared memory, the changes made will immediately affect any other process that can access the same piece of shared memory. Shared memory allows two unrelated processes to access the same logical memory, a very efficient way to share and transfer data between two running processes. The data stream storage area is an area for storing data streams in the shared memory, and the division of the data stream storage area may be a logic division, that is, the data stream storage area is logically divided in the shared memory to store the target data streams acquired by the multiplexing process. In the actual physical address of the shared memory, the data stream storage area may be shared with the actual physical address of other areas, and the shared memory is logically divided into different storage areas, so as to store different data in the different storage areas, for example, store the data stream acquired by the multiplexing process in the data stream storage area. The storage information is information related to storage of the target data stream, and may include, but not limited to, storage locations including the target data stream, and attribute information such as data size, write time, check information, data sequence number, and the like of the target data stream. The written target data stream can be quickly and accurately read from the data stream storage area of the shared memory through the storage information, for example, the storage position of the target data stream can be determined from the data stream storage area of the shared memory based on the storage position in the storage information, and the read target data stream can be verified based on the attribute information in the storage position, so that the accuracy of the read target data stream is ensured.
Specifically, after the target data stream to be distributed is obtained through the multiplexing process, the server writes the target data stream into the shared memory through the multiplexing process, specifically, the multiplexing process can determine a data stream storage area of the shared memory, and writes the target data stream into the data stream storage area to obtain storage information of the target data stream. For example, the server determines a storage location in the data stream storage area where writing is required by the multiplexing process, and writes the target data stream in the storage location, and in the process of writing the target data stream, obtains storage information of the target data stream by the multiplexing process, for example, obtains storage location and attribute information of the target data stream, where the attribute information may include, but is not limited to, information related to storage of the data stream including data size, writing time, verification information, data sequence number, and the like.
And 206, writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process.
The data stream information area is an area for storing information of the data stream in the shared memory, and the data stream information area can be logically divided, that is, the data stream storage area and the data stream information area in the shared memory can share actual physical addresses, but the data stream storage area and the data stream information area are logically divided into different areas to form a two-stage structure on the shared memory. Specifically, the shared memory may be divided into two levels of structures, and the first level shared memory may be a data stream information area for storing information of a data stream, such as storage information capable of storing the data stream; the second-level shared memory may be a data stream storage area for storing the data stream itself, that is, storing a specific data stream, and the first-level shared memory and the second-level shared memory may be associated based on the storage information. When reading from the shared memory, the process may first read the storage information of the target data stream from the first level shared memory, and based on the storage information of the target data stream, read the target data stream from the second level shared memory.
Specifically, after obtaining the storage information of the target data stream, the server further determines the data stream information area of the shared memory through the multiplexing process, and writes the storage information into the data stream information area of the shared memory, so that the process can read the target data stream obtained by the multiplexing process from the data stream information area of the shared memory, and the reuse of the target data stream is realized in the shared memory. In addition, the multiplexing process writes the obtained target data stream into the shared memory to realize multiplexing of the obtained target data stream, and the multiplexing process can also send the obtained target data stream to a terminal associated with the multiplexing process, wherein the terminal associated with the multiplexing process comprises, but is not limited to, a mobile phone, a computer, intelligent voice interaction equipment, intelligent home appliances, vehicle-mounted terminals, aircrafts and the like, so that data stream obtaining of the associated terminal is realized.
In a specific application, the multiplexing process is a process which is started by the server and associated with the terminal after the terminal is connected with the server, and the multiplexing process can acquire a target data stream from a data stream network and send the target data stream to the terminal; meanwhile, the multiplexing process can write the obtained target data stream into the data stream storage area of the shared memory, and write the storage information of the target data stream into the data stream information area of the shared memory, so that other processes started by the server can obtain the target data stream from the shared memory for distribution, thereby realizing the recycling of the target data stream, avoiding the repeated data stream obtaining processing of other processes, and improving the processing efficiency of the data stream. In addition, the multiplexing process may also be a newly-built process, specifically, after the terminal is connected to the server, the server may start an association process associated with the terminal, through which the server may forward the uplink data stream uploaded by the terminal, for example, the server may send the uplink data stream of the terminal to the data stream network through the association process, and through newly-built multiplexing process, establish an association relationship between the multiplexing process and the terminal, so that the multiplexing process obtains the downlink target data stream from the data stream network, writes the target data stream into the shared memory, and simultaneously sends the target data stream to the terminal, thereby implementing distribution processing of the target data stream.
Step 208, reading the storage information from the data stream information area through at least one distribution process, and obtaining the target data stream from the data stream information area according to the read storage information.
The distribution process is a designated process for realizing data stream distribution processing, namely the distribution process can read a stored data stream from a shared memory and send the read data stream to a terminal associated with the distribution process so as to realize communication of the terminal associated with the distribution process. Compared with the multiplexing process, the distributing process can directly read the data stream from the shared memory and send the read data stream to the associated terminal. The distribution process does not directly acquire the data stream, but reuses the data stream acquired by the multiplexing process based on the shared memory. For the shared memory, the multiplexing process is a writer of the shared memory, namely, the multiplexing process is a producer and is responsible for acquiring the target data stream and writing the acquired target data stream into the shared memory; the distribution process is a reader of the shared memory, namely the distribution process is a consumer and is responsible for reading the target data stream from the shared memory and sending the target data stream to the associated terminal, so that a data stream processing mode of one-write-multiple-read and multiple-consumer of a single producer is realized.
Specifically, the distribution process includes at least one, the distribution process is associated with a terminal connected to the server, after the terminal is connected to the server, the server may start a process associated with the connected terminal, designate the process as a distribution process, and the number of terminals connected to the server is plural, so that the corresponding distribution process is plural. The server determines a data stream information area of the shared memory through at least one distribution process, reads storage information from the data stream information area, determines the data stream storage area of the shared memory based on the read storage information after the distribution process obtains the storage information, and reads the target data stream written by the multiplexing process from the data stream storage area. The distribution process can quickly and accurately obtain the target data stream written by the multiplexing process from the shared memory through the secondary reading processing of the data stream information area and the data stream storage area so as to realize the recycling of the target data stream.
And step 210, transmitting the target data stream obtained from the shared memory to the terminal associated with the distribution process through at least one distribution process.
Specifically, after the target data stream written by the multiplexing process is read from the shared memory by the distribution process, the server determines the terminal associated with the distribution process, and sends the target data stream read from the shared memory to the terminal associated with the distribution process by the distribution process, so that the reuse of the target data stream acquired by the multiplexing process is realized, and the efficient communication processing of the terminal associated with the distribution process is realized.
In a specific application, as shown in fig. 3, the terminals 1 to N are respectively connected to a server, a process associated with the terminal is started in the server to implement communication of the terminal, specifically, for the terminal 1, the server establishes the process 1 to implement communication processing of the terminal 1, and for the terminal N, the server starts the process N to implement communication processing of the terminal N. The server can be connected with the data stream network, so that the downlink data stream which needs to be issued to each terminal is obtained from the data stream network, and meanwhile, the server can also send the uplink data stream sent by each terminal to the data stream network, so that the communication of each terminal in the data stream network is realized. For uplink data processing, each process acquires an uplink data stream of each associated terminal, and sends the acquired uplink data stream to a data stream network. For the processing procedure of the downlink data stream, the server may designate a multiplexing process from the processes associated with each terminal, if the server designates the process 1 associated with the terminal 1 as a multiplexing process, the server obtains the target data stream to be distributed from the data stream network through the process 1, sends the target data stream to the terminal 1, and writes the obtained target data stream into the shared memory preset by the server through the process 1. When writing the target data stream, the server can determine the data stream storage area of the shared memory through the process 1, write the target data stream into the data stream storage area to obtain the storage information of the target data stream, and write the storage information into the data stream information area of the shared memory through the multiplexing process, so that the obtained target data stream is written into the shared memory with a two-stage structure, and the high-efficiency and accurate reading and writing processing of the target data stream is ensured. For other N-1 processes except the process 1, the process can be used as a distribution process, the read target data streams obtained and written by the process 1 from the data stream network are respectively read from the shared memory of the server and sent to the respective associated terminals, so that the processing of the downlink data streams of the terminals is realized, and the efficient communication of the terminals in the data stream network is realized. In the processes associated with each terminal in the server, only the process 1 is required to perform the process of acquiring the target data stream, and other processes can directly read the target data stream written by the process 1 obtained from the shared memory, so that the number of processes for directly acquiring the data stream is reduced, the processing cost of the server for repeatedly acquiring the data stream is reduced, and the processing efficiency of the data stream is improved.
In the above data stream processing method, the multiplexing process writes the obtained target data stream to be distributed into the data stream storage area of the shared memory, writes the obtained storage information of the target data stream into the data stream information area of the shared memory, sends the target data stream to the terminal associated with the multiplexing process, reads the storage information from the data stream information area through at least one distribution process, obtains the target data stream from the data stream storage area according to the read storage information, and sends the target data stream obtained from the shared memory to the terminal associated with the distribution process through the distribution process. In the data stream processing process, the obtained target data stream is written into a data stream storage area of the shared memory through a multiplexing process associated with the terminal, and storage information of the target data stream is written into a data stream information area of the shared memory, so that the target data stream stored in the shared memory can be quickly read and sent to the associated terminal through a distribution process, when a plurality of terminals need to obtain the same data stream, the number of processes for directly obtaining the data stream is reduced, the data stream is directly read from the shared memory through the distribution process for distribution, and the processing efficiency of the data stream is improved.
In one embodiment, writing the target data stream into the data stream storage area of the shared memory through the multiplexing process, and obtaining the storage information of the target data stream, including: determining a preset data stream storage area of the shared memory through a multiplexing process; writing the target data stream in a data stream storage location in the data stream storage area; and obtaining the storage information of the target data stream according to the data stream storage position.
The shared memory is the same section of physical memory shared among different processes, and a plurality of threads running on the server can access the shared memory to obtain data in the shared memory. A multiplexed process is a producer of shared memory, i.e., the multiplexed process writes data into the shared memory for consumption by other processes as consumers reading from the shared memory. The data stream storage area is an area for storing data streams in the shared memory, and the data stream storage position is a specific position where the data streams need to be written in the data stream storage area, namely, the target data streams acquired by the multiplexing process are written in the data stream storage position of the data stream storage area. The multiplexing process can determine a data stream storage position needing to be written in the data stream storage area, write the target data stream in the data stream storage position, and obtain storage information of the target data stream according to the data stream storage position, so that when data is read from the shared memory, the target data stream can be read from the data stream storage area of the shared memory based on the storage information.
Specifically, when the target data stream is written into the shared memory through the multiplexing process, the server determines a preset shared memory through the multiplexing process, and the shared memory may be preset by the server, for example, a physical memory address is designated as the shared memory. The shared memory comprises a data stream storage area for storing specific data streams, and the data stream storage area can be an area divided based on actual physical memory addresses or an area divided based on logic. The server determines the data stream storage area in the shared memory through the multiplexing process and determines the data stream storage position in the data stream storage area, for example, the server can determine the latest writing position in the data stream storage area through the multiplexing process, thereby determining the data stream storage position. After determining the data stream storage location, the server writes the target data stream in the data stream storage location by a multiplexing process. In a specific application, the data stream storage location may include different fields, such as a data size field, a data sequence number field, a write time field, and the like, and the multiplexing process may write the target data stream into the data field of the data stream storage location. In addition, the server obtains the storage information of the target data stream according to the data stream storage position through the multiplexing process, and the storage information can comprise the data stream storage position, so that the distribution process can read the target data stream written by the obtained multiplexing process from the data stream storage area according to the data stream storage position in the storage information.
In this embodiment, the server writes the obtained target data stream into the data stream storage location of the shared memory through the multiplexing process, and obtains the storage information of the target data stream according to the data stream storage location, so that the obtained target data stream is written into the data stream storage location of the shared memory through the multiplexing process, so that the distribution process can read the obtained target data stream from the data stream storage location of the shared memory according to the data stream storage location in the storage information, when a plurality of terminals need to obtain the same data stream, the number of processes for directly obtaining the data stream is reduced, and the data stream is directly read from the shared memory through the distribution process for distribution, thereby improving the processing efficiency of the data stream.
In one embodiment, writing a target data stream in a data stream storage location in a data stream storage area includes: determining a data stream storage position in a data stream storage area and attribute information of a target data stream; the target data stream and the attribute information are written in the data stream storage location.
The data stream storage position is a specific position where the data stream needs to be written in the data stream storage area, namely, the target data stream acquired by the multiplexing process is written in the data stream storage position of the data stream storage area. The attribute information is information related to a data attribute of the target data stream, and may be used to describe the target data stream, and may include, but not limited to, information including a data size, a writing time, verification information, a data sequence number, and the like of the target data stream. The type of the attribute information of the target data stream can be flexibly set according to actual needs so as to accurately describe the target data stream, thereby being beneficial to confirming the target data stream and ensuring the accuracy of the target data stream.
Specifically, when writing the target data stream in the data stream storage location of the data stream storage area, the server may determine the data stream storage location in the data stream storage area through the multiplexing process, and the data stream storage location may be the latest writing location in the data stream storage area, that is, may be updated according to the last writing location. For example, the last writing location is numbered 3, the latest writing location after updating the writing location may be 5, and then the data stream storage location in the data stream storage area may be determined to be 5, that is, the target data stream obtained by the multiplexing process is written in the storage location of the number 5. The data stream storage location may include an attribute information field and a data field, where the data field is used to write the data stream, and the attribute information field is used to write attribute information related to the data stream, such as a data size, a writing time, and verification information of the data stream. The attribute information of the target data stream can be obtained after the server analyzes the target data stream through the multiplexing process, for example, the data size of the target data stream can be analyzed, the writer identification and writing time of the target data stream are determined, and the verification information of the target data stream is calculated. After determining the data stream storage position in the data stream storage area and the attribute information of the target data stream, the server writes the target data stream and the attribute information into the data stream storage position in the data stream storage area through the multiplexing process, so that the target data stream is written into the data stream storage area of the shared memory.
Further, according to the data stream storage location, obtaining storage information of the target data stream includes: and obtaining the storage information of the target data stream according to the data stream storage position and the attribute information.
Specifically, the server obtains the storage information of the target data stream according to the data stream storage position and the attribute information of the target data stream through the multiplexing process. The storage information of the target data stream includes a storage position of the target data stream in the data stream storage area, that is, a data stream storage position, and further includes attribute information of the target data stream, such as a data size, a writing time, verification information, a data stream sequence number, a writer identifier, and the like of the target data stream.
In this embodiment, the server writes the target data stream and attribute information of the target data stream in the data stream storage location of the data stream storage area through the multiplexing process, and obtains the storage information of the target data stream based on the data stream storage location and the attribute information, so that the distribution flow can read the target data stream from the data stream storage area of the shared memory according to the data stream storage location in the storage information, and verify the target data stream according to the attribute information in the storage information, thereby ensuring the accuracy of the read data stream.
In one embodiment, writing a target data stream in a data stream storage location in a data stream storage area includes: determining a data stream storage position in the data stream storage area according to the data stream storage position identification of the data stream storage area; acquiring a write permission for a data stream storage position; based on the write rights, the target data stream is written in the data stream storage location.
The data stream storage location identifier may be location identifier information of the data stream storage location, and may specifically be location identifier information such as a location number, a physical address, and the like. The data stream storage location identifier corresponds to a data stream storage location in the data stream storage area, from which the corresponding data stream storage location can be uniquely determined. In a specific application, the data stream storage location may be the latest writing location in the data stream storage area, and then the data stream storage location identifier may be the location identifier of the latest writing location in the data stream storage area. The write permission refers to an operation permission for executing writing to a data stream storage location in the data stream storage area, and only a process with the write permission can write to the data stream storage location, so that the conflict of the processes to write to the data stream storage location is avoided.
Specifically, the server obtains the data stream storage location identifier of the data stream storage area through the multiplexing process, and specifically can query the writing information of the data stream storage area, so as to extract the data stream storage location identifier of the latest writing location from the writing information. The server determines the data stream storage position in the data stream storage area based on the data stream storage position identification through the multiplexing process, wherein the data stream storage position is a storage position in the shared memory, and the target data stream needs to be written in the storage position. The server obtains the write-in authority for the data stream storage position through the multiplexing process, so that the multiplexing process has the write-in authority for the data stream storage position. In a specific application, the server can acquire the writing permission of the data stream storage position based on a lock-free algorithm through a multiplexing process. For example, a CAS (Compare And Swap) algorithm may be executed by a multiplexing process, so that the multiplexing process obtains the write permission of the data stream storage location. Wherein, the CAS algorithm comprises 3 operands, a memory value V, an old expected value A and a new value B to be modified; if and only if the expected value A and the memory value V are the same, the memory value V is modified to B, otherwise nothing is done. Based on the CAS algorithm, the expected value A in the operand of the multiplexing process and the memory value V of the data stream storage location can be made the same, so that the memory value V of the data stream storage location can be modified to write the target data stream. After the multiplexing process has the write-in authority of the data stream storage position, the server writes the acquired target data stream into the data stream storage position based on the acquired write-in authority through the multiplexing process, so that the target data stream is written into the shared memory.
In this embodiment, the server determines, through the multiplexing process, a data stream storage location in the data stream storage area according to the data stream storage location identifier of the data stream storage area, and after obtaining the write permission of the data stream storage location, writes the obtained target data stream into the data stream storage location through the multiplexing process based on the write permission, and can accurately determine the data stream storage location to be written through the data stream storage location identifier, and ensures that the multiplexing process and other processes cannot generate write conflicts on the data stream storage location through the write permission, thereby ensuring the processing efficiency of writing the target data stream into the shared memory, and further ensuring the processing efficiency of the data stream.
In one embodiment, writing the stored information into the data stream information area of the shared memory by the multiplexing process includes: determining a data flow information area of the shared memory through a multiplexing process; determining an information storage position in the data stream information area according to the information storage position identification of the data stream information area; the stored information is written in the information storage location.
The data stream information area is an area in the shared memory for storing information of the data stream, and the information storage location identifier is location identifier information of an information storage location in the data stream information area, where the storage information needs to be written, and specifically may be identifier information such as a location number, a physical address, and the like. The information storage location identifier corresponds to an information storage location in the data stream information area, and the corresponding information storage location can be uniquely determined from the information storage area based on the information storage location identifier. In a specific application, the information storage location may be the latest writing location in the information storage area, and the information storage location identifier may be the location identifier of the latest writing location in the information storage area. The storage information is information related to storage of the target data stream, and may include, but not limited to, storage locations including the target data stream, and attribute information such as data size, write time, check information, data sequence number, and the like of the target data stream.
Specifically, the multiplexing process writes the target data stream into the data stream information area of the shared memory, after obtaining the storage information of the target data stream, the server determines the data stream information area of the shared memory through the multiplexing process, and the data stream information area can be an area of the shared memory which is based on the information of the data stream and is divided by logic. The server determines the information storage position identification of the data stream information area through the multiplexing process, and determines the information storage position in the data stream information area based on the information storage position identification, wherein the information storage position is the position in the data stream information area where the storage information needs to be written. The server writes the storage information of the target data stream into the information storage position of the data stream information area through the multiplexing process, so that the storage information of the target data stream is written into the data stream information area of the shared memory, so that the distribution process can read the storage information from the data stream information area of the shared memory, and the target data stream is obtained from the data stream storage area of the shared memory based on the read storage information for distribution.
In this embodiment, the server determines, by using the multiplexing process, an information storage location in the data stream information area according to the information storage location identifier of the data stream information area, and writes the obtained storage information of the target data stream into the information storage location by using the multiplexing process, so that the distribution process can read the storage information from the data stream information area of the shared memory, and read the target data stream from the data stream storage area of the shared memory based on the read storage information for distribution. When a plurality of terminals need to acquire the same data stream, the number of processes for directly acquiring the data stream is reduced, the data stream is directly read from the shared memory for distribution through the distribution process, and the processing efficiency of the data stream is improved.
In one embodiment, as shown in fig. 4, the process of reading the target data stream, that is, reading the storage information from the data stream information area through at least one distribution process, and obtaining the target data stream from the data stream storage area according to the read storage information, includes:
step 402, determining, by at least one distribution process, a data stream information area of the shared memory.
The distribution process is a designated process for realizing data stream distribution processing, namely the distribution process can read a stored data stream from a shared memory and send the read data stream to a terminal associated with the distribution process so as to realize communication of the terminal associated with the distribution process. The number of distribution processes is related to the number of terminals to which the connection server needs to read data from the shared memory. If, for each terminal connected to the server, the server starts a process associated with the terminal, each terminal is associated with a corresponding process, and the number of distribution processes is the same as the number of terminals that need to read data from the shared memory. Compared with the multiplexing process, the distributing process can directly read the data stream from the shared memory and send the read data stream to the associated terminal. The distribution process does not directly acquire the data stream, but reuses the data stream acquired by the multiplexing process based on the shared memory. The data stream information area is an area in the shared memory for storing information of the data stream, the data stream information area stores storage information of the data stream, and a required data stream can be read from the data stream storage area of the shared memory based on the storage information of the data stream.
Specifically, the server determines a preset shared memory through at least one distribution process, and determines a data stream information area of the shared memory, so as to read storage information from the data stream information area of the shared memory, and read a target data stream from the shared memory based on the storage information.
Step 404, according to the information storage location identification of the data stream information area, the storage information is read from the data stream information area.
The information storage location identifier is location identifier information of an information storage location in which storage information is newly written in the data stream information area, and may specifically be location identifier information such as a location number and a physical address. The information storage location identifier corresponds to an information storage location in the data stream information area, and the corresponding information storage location can be uniquely determined from the information storage area based on the information storage location identifier. The storage information is information related to the storage of the target data stream.
Specifically, the server determines, by the distribution process, an information storage location identifier of the data stream information area, the information storage location identifier corresponding to an information storage location in the data stream information area. The distribution process may read the stored information from the data stream information area based on the information storage location identification of the data stream information area. The server can determine the information storage position from the data stream information area based on the information storage position identification through the distribution process, wherein the information storage position is the storage position for storing information and is also the position which needs to be read currently, and the distribution process reads the stored information from the determined information storage position.
And step 406, obtaining the target data stream from the data stream storage area of the shared memory according to the read storage information.
The data stream storage area is an area for storing data streams in the shared memory, namely, the data streams acquired and written by the multiplexing process are stored in the data stream storage area. Specifically, the server reads the target data stream from the data stream storage area of the shared memory based on the storage information read from the data stream information area through the distribution process. The server can determine the data stream storage position in the data stream storage area of the shared memory from the read storage information through the distribution process, and the distribution process reads the target data stream from the data stream storage position in the data stream storage area.
In this embodiment, the server reads the storage information from the data stream information area according to the information storage location identifier of the data stream information area through at least one distribution process, and obtains the target data stream from the data stream storage area of the shared memory based on the read storage information, so that when multiple terminals need to obtain the same data stream, the number of processes for directly obtaining the data stream is reduced, and the data stream is directly read from the shared memory for distribution through the distribution process, thereby improving the processing efficiency of the data stream.
In one embodiment, obtaining the target data stream from the data stream storage area of the shared memory according to the read storage information includes: performing storage information verification on the read storage information to obtain a storage information verification result; when the verification result of the stored information is that the verification is passed, reading the data stream from the data stream storage area of the shared memory according to the stored information; and when the reading result obtained by reading the data stream passes the data stream verification, obtaining the target data stream according to the reading result.
The storage information verification result is a verification result obtained by verifying the read storage information, and represents validity of the storage information read from the data stream information area of the shared memory, namely whether the read storage information is valid or not. The data stream reading is a process of reading from the data stream storage area of the shared memory according to the storage information to obtain the data stream when the obtained storage information is valid. The data stream verification shows that the data stream is effectively read from the data stream storage area of the shared memory, so that the target data stream can be obtained according to the data stream.
The server performs storage information verification on the read storage information through a distribution process, and can perform storage information verification based on verification information in the storage information to verify validity of the storage information. When the verification result of the stored information is that the verification is passed, the read stored information is legal and effective, the server reads the data stream from the data stream storage area of the shared memory according to the stored information through a distribution process, and concretely, the server determines the data stream storage position based on the stored information through the distribution process and reads the data stream from the data stream storage position of the data stream storage area to obtain a reading result. When the data stream reading result passes the data stream verification, the server can verify the data stream of the data stream reading result through the distribution process to verify the validity of the data stream reading result, and when the data stream verification is passed, the server obtains the target data stream according to the reading result through the distribution process, thereby realizing the accurate and rapid reading of the target data stream from the shared memory through the distribution process, and carrying out the distribution processing on the target data stream.
If the stored information verification result is that the verification is not passed, or the read result obtained by reading the data stream fails the data stream verification, it indicates that the stored information read by the distribution process is illegal, the read stored information is invalid, or the data stream read by the distribution process is illegal, the read result can be discarded and the read can be performed again, if the stored information can be read from the data stream information area by the distribution process according to the information storage location identifier of the data stream information area, the next read processing can be performed.
In this embodiment, when the verification result of the storage information of the read storage information is that the verification is passed, the server reads the data stream from the data stream storage area of the shared memory according to the storage information through the distribution process, if the read result obtained by reading the data stream passes the data stream verification, the target data stream is obtained according to the read result, so that the target data stream is accurately and quickly read from the shared memory through the distribution process, the target data stream is distributed, and when a plurality of terminals need to acquire the same data stream, the number of processes for directly acquiring the data stream is reduced, and the data stream is directly read from the shared memory through the distribution process for distribution, thereby improving the processing efficiency of the data stream.
In one embodiment, performing storage information verification on the read storage information to obtain a storage information verification result, including: extracting storage information to be checked and first check reference information from the read storage information; generating storage matching information according to storage information verification conditions and storage information to be verified; and checking and matching the stored matching information with the first checking reference information to obtain a stored information checking result.
The storage information to be checked is information to be checked in the storage information, and specifically, the information can be corresponding to each field such as data size, writing time, frame number, data storage position and the like; the first verification reference information is the reference information for verification in the storage information, and the storage information to be verified can be verified based on the first verification reference information, so that the storage information verification processing of the storage information is realized, and the validity of the storage information is ensured. In a specific application, the first verification reference information may be verification reference information obtained after performing operation processing on information written in a field corresponding to the storage information to be verified by the multiplexing process. And when the multiplexing process writes in the corresponding field of the storage information to be verified, summing according to the written information to obtain first verification reference information, wherein the storage information of the target data stream comprises the first verification reference information and the written information. The stored information verification condition includes a processing mode of processing the stored information to be verified to obtain stored matching information for verification matching, for example, the stored information verification condition may include an algorithm of the stored information to be verified, and may specifically be various processing modes such as summation, product seeking, and the like. And processing the storage information to be checked through the storage information checking condition, so that storage matching information corresponding to the storage information to be checked can be generated, and checking and matching can be carried out between the storage matching information and the first checking reference information through the storage matching information, thereby realizing the storage information checking processing of the storage information and determining the legal validity of the read storage information.
Specifically, the server extracts the storage information to be verified and the first verification reference information from the read storage information through a distribution process. The storage information may include data of different fields, specifically may include data of a reference field and data of a non-reference field, and the distribution process may extract first verification reference information corresponding to the reference field and storage information to be verified corresponding to the non-reference field from the storage information. The server acquires preset storage information verification conditions through a distribution process, wherein the storage information verification conditions comprise a processing mode of storage information to be verified when storage information is subjected to storage information verification processing, so as to generate storage matching information corresponding to the storage information to be verified. And the server generates storage matching information according to the storage information to be checked and the acquired storage information checking condition through a distribution process. The server can sum the storage information to be checked through the storage information checking condition by the distribution process to obtain the storage matching information. And the server performs verification matching on the stored matching information and the first verification reference information through a distribution process to obtain a stored information verification result. If the stored matching information is consistent with the first verification reference information in verification matching, the read stored information is legal and effective, and the data stream reading can be continuously performed based on the stored information; if the stored matching information is inconsistent with the first verification reference information, the stored information obtained by reading is illegal, and the stored information is invalid, the reading can be abandoned, and the reading processing can be carried out again, so that the accuracy of the data stream reading is ensured.
In this embodiment, the server extracts the storage information to be checked and the first check reference information from the read storage information through the distribution process, generates storage matching information according to the storage information to be checked according to a preset storage information check condition, and performs check matching on the storage matching information and the first check reference information, so as to realize storage information check of the storage information, obtain a storage information check result, ensure legal and effective of the read storage information, and improve accuracy of data stream reading.
In one embodiment, when a read result obtained by reading the data stream passes the data stream verification, obtaining the target data stream according to the read result includes: determining data flow information to be checked, first check reference information and second check reference information according to a reading result obtained by data flow reading; according to the data stream information to be checked, respectively generating storage matching information and data stream matching information according to storage information checking conditions and data stream checking conditions; respectively checking and matching the stored matching information with the first checking reference information, and checking and matching the data stream matching information with the second checking reference information to obtain a data stream checking result; and when the data stream verification result is that the verification passes, obtaining a target data stream according to the reading result.
The data stream information to be checked is information to be checked, and specifically may include data corresponding to each field such as data size, writing time, frame number, data storage position, read data stream, etc.; compared with the stored information to be verified in the stored information, the data stream information to be verified comprises the read data stream itself besides the stored information to be verified in the stored information. The first verification reference information is used for verifying the stored information, and the information corresponding to the stored information in the data stream information to be verified can be verified based on the first verification reference information; the second checking reference information is the reference information for checking the data stream information to be checked. In a specific application, the first verification reference information may be verification reference information obtained after performing operation processing on information written in a field, which is the same as the storage information, in the data stream information to be verified in the multiplexing process. The second verification reference information may be verification reference information obtained after performing operation processing on information written in a field corresponding to the data stream information to be verified by the multiplexing process. By the first checking reference information and the second checking reference information, the storage information covered in the data stream information to be checked and the data stream itself can be checked, and the accuracy of data stream checking is ensured.
The stored information verification conditions and the data stream verification conditions respectively comprise a processing mode for processing the data stream information to be verified so as to obtain matching information for verification and matching. The stored information verification conditions and the data stream verification conditions may include different processing manners, and may also be specific to different data types in the data stream information to be verified. Specifically, the storage information verification condition may include a processing manner of processing the same information as the storage information to be verified in the data stream information to be verified to obtain storage matching information for verification matching, for example, may include an algorithm of the same information as the storage information to be verified in the data stream information to be verified, and may specifically be various processing manners such as summation, product seeking, and the like. And processing the information which is the same as the stored information in the data stream information to be checked through the stored information checking condition, so that the stored matching information corresponding to the information which is the same as the stored information in the data stream information to be checked can be generated, and checking and matching can be performed with the first checking reference information through the stored matching information, thereby realizing the checking processing of the information which is the same as the stored information in the data stream information to be checked. The data stream verification condition may include processing all information of the data stream information to be verified, specifically including the same information as the stored information and the stored data stream itself, so as to obtain a processing mode of the data stream matching information for verification matching, for example, may include an algorithm of the data stream information to be verified, and specifically may be various processing modes such as summation, product seeking, and the like. And processing the data stream information to be checked through the data stream checking condition, so that data stream matching information corresponding to the data stream information to be checked can be generated, and checking and matching can be performed between the data stream matching information and the second checking reference information through the data stream matching information, thereby realizing checking processing of the data stream information to be checked.
Specifically, the server determines data stream information to be checked, first check reference information and second check reference information from a read result obtained by reading the data stream through a distribution process. The data stream information to be checked comprises the same information as the stored information to be checked in the reference information and the stored data stream itself. The server acquires preset storage information verification conditions and data stream verification conditions through a distribution process, wherein the storage information verification conditions comprise the same information as the storage information to be verified in the data stream information to be verified, and when the verification processing is carried out on the same information, the storage matching information corresponding to the data stream information to be verified is generated; the data stream verification condition comprises a processing mode of all information of the data stream information to be verified when all information in the data stream information to be verified is verified, so as to generate data stream matching information corresponding to the data stream information to be verified.
And the server generates storage matching information and data stream matching information according to the data stream information to be checked through a distribution process and the obtained storage information check condition and the data stream check condition respectively. The server can carry out summation operation on the information which is the same as the stored information in the stored information to be checked through the distribution process and the stored information checking condition to obtain the stored matching information; and carrying out summation operation on the storage information to be checked through the data stream checking condition, and matching the data stream with the information. And the server respectively performs verification and matching on the stored matching information and the first verification reference information through a distribution process, and performs verification and matching on the data stream matching information and the second verification reference information to obtain a data stream verification result. If the stored matching information is matched with the first check reference information in a check mode, and the data stream matching information is matched with the second check reference information in a check mode, the data stream obtained through reading is valid, the data stream checking result is that the check is passed, and the server can obtain the target data stream written into the shared memory by the multiplexing process according to the reading result through the distributing process.
In this embodiment, the server determines, according to the to-be-verified data stream information, the first verification reference information and the second verification reference information according to the read result obtained by reading the data stream by the distribution process, generates, according to the to-be-verified data stream information, the stored matching information and the data stream matching information according to the stored information verification condition and the data stream verification condition, respectively performs verification matching on the stored matching information and the first verification reference information, performs verification matching on the data stream matching information and the second verification reference information, and obtains a data stream verification result.
In one embodiment, reading the stored information from the data stream information area according to the information storage location identification of the data stream information area comprises: acquiring an information storage position identifier of a data stream information area; determining a current position identifier of an information storage position read currently by a distribution process; determining a target position identifier from the information storage position identifiers of the data stream information area according to the current position identifier; the stored information is read from the data stream information area based on the destination location identification.
The information storage location identifier is location identifier information of an information storage location in which storage information needs to be written in the data stream information area, and may specifically be identifier information such as a location number, a physical address, and the like. The current location identity is a location identity of an information storage location that is currently being read and that is to be read. The target location identifier is a location identifier of an information storage location to be read, which is redetermined according to the current location identifier and the information storage location identifier in the data stream information area.
Specifically, when the storage information is read, the server acquires the information storage position identification of the data stream information area through the distribution process, and determines the current position identification of the information storage position currently read by the distribution process. And the server determines the target position identification from the information storage position identifications of the data stream information area according to the current position identification through a distribution process. In a specific application, the information storage location identifier of the data stream information area may be a location identifier of a latest writing location in the data stream information area, and the server compares the current location identifier with the information storage location identifier of the data stream information area through the distribution process, thereby determining a difference between the information storage location currently read by the distribution process and the latest writing location in the data stream information area, determining a target location identifier based on the difference, and reading the storage information from the data stream information area by the distribution process based on the target location identifier.
In particular, when the information storage position currently read by the distribution process is the same as the latest writing position in the data stream information area, it indicates that the distribution process currently reads the latest writing data in the data stream information area, and then it can determine that the target position identifier is the current position identifier or the information storage position identifier of the data stream information area. If the current read information storage position of the distribution process is different from the latest writing position in the data stream information area and the difference does not meet the updating condition, if the difference does not exceed the preset data length, the target position mark can be determined to be the current position mark, and the current position mark is read from the data stream information area, so that the integrity of the read data stream is ensured. If the current read information storage position of the distribution process is different from the latest writing position in the data stream information area and the difference meets the updating condition, if the difference exceeds the preset data length, the target position mark can be determined to be the information storage position mark of the data stream information area, and the information storage position mark of the data stream information area is read from the data stream information area, so that the real-time performance of the read data stream is ensured.
In this embodiment, the server determines, through the distribution process, the target location identifier according to the current location identifier of the information storage location currently read by the distribution process and the information storage location identifier of the data stream information area, and reads the storage information from the data stream information area based on the target location identifier, so that the real-time performance and the integrity of the data when the data stream is read can be balanced.
In one embodiment, the target data stream is a media data stream, and the obtaining the target data stream to be distributed through the multiplexing process includes: when a first terminal accesses a media network, a media data stream is acquired from the media network through a multiplexing process associated with the first terminal.
The target data stream is a media data stream, and the media data stream may include at least one data stream of an audio data stream, a video data stream, an image data stream, and a text data stream, and in different application scenarios, the media data stream may have different types of data forms. The media network is a network for accessing the terminal to communicate, and the terminal performs media data stream transmission through the media network so as to realize network communication. The first terminal is a terminal that first accesses the media network. Specifically, when the server detects that the first terminal accesses the media network, the server may start a process associated with the first terminal, select the process associated with the first terminal as a multiplexing process, and acquire a media data stream from the media network through the multiplexing process.
Further, sending the target data stream to a terminal associated with the multiplexing process, including: the media data stream is transmitted to the first terminal through a multiplexing process.
Specifically, when the target data stream is sent to the terminal associated with the multiplexing process, the server may send the obtained media data stream to the first terminal through the multiplexing process, so that the first terminal directly obtains the media data stream obtained by the multiplexing process from the media network.
Further, by at least one distribution process, reading the storage information from the data stream information area, and obtaining the target data stream from the data stream information area according to the read storage information, comprising: when at least one second terminal accesses the media network, the stored information is read from the data stream information area by a distribution process associated with the second terminal, and the media data stream is obtained from the data stream storage area according to the read stored information.
The second terminal is a terminal accessing the media network after the first terminal. Specifically, when the server detects that at least one second terminal accesses the media network after the first terminal, the server may start a process associated with the second terminal, and set the process associated with the second terminal as a distribution process. The server reads the storage information from the data stream information area through a distribution process associated with the second terminal, and obtains the media data stream from the data stream information area according to the read storage information.
Further, the target data stream obtained from the shared memory is sent to the terminal associated with the distribution process through at least one distribution process, including: and sending the media data stream obtained from the shared memory to the second terminal through the distribution process.
Specifically, after the server obtains the media data stream from the shared memory through the distribution process associated with the second terminal, the server sends the media data stream obtained from the shared memory to the second terminal through the distribution process, so that the distribution process associated with the second terminal obtains the media data stream from the shared network without repeating pulling from the media network.
In one specific application, as shown in fig. 5, the first terminal connects with a server to access a media network to communicate over the media network. After the server is connected with the first terminal, starting a process 1 associated with the first terminal, designating the process 1 as a multiplexing process, pulling a media data stream from a media network by the first process, sending the pulled media data stream to the first terminal, and writing the pulled media data stream into a preset shared memory. As shown in fig. 6, the second terminal is also connected to the server after the first terminal to access the same media network, after the server is connected to the second terminal, the process 2 associated with the second terminal is started, and the process 2 is designated as a distribution process, the process 2 may read the media data stream pulled and written from the media network by the process 1 from the shared memory, and send the read media data stream to the second terminal, and in addition, the process 2 may also send the uplink data stream sent by the second terminal to the media network. As shown in fig. 7, the third terminal is also connected to the server after the first terminal and the second terminal, so as to access to the same media network, after the server is connected to the third terminal, the process 3 associated with the third terminal is started, and the process 3 is designated as a distribution process, and the process 3 may read the media data stream pulled and written from the media network by the process 1 from the shared memory, and send the read media data stream to the third terminal, and in addition, the process 3 may also send the uplink data stream sent by the third terminal 3 to the media network. Further, other terminals can be continuously accessed, and for each terminal accessed later, the associated process is set as a distribution process, so that the media data stream is acquired from the shared memory through the distribution process and is sent to the corresponding terminal.
In this embodiment, for a first terminal that first accesses a media network, a server obtains a media data stream from the media network through a multiplexing process associated with the first terminal, and sends the media data stream to the first terminal through the multiplexing process; for a second terminal accessing the media network after the first terminal, the server reads the storage information from the data stream information area through a distribution process associated with the second terminal, and obtains a media data stream from the data stream storage area according to the read storage information, and transmits the obtained media data stream to the second terminal. In the media data stream processing process, the acquired media data stream is written into a data stream storage area of the shared memory through a multiplexing process associated with a first terminal which is accessed before, and storage information of a target data stream is written into a data stream information area of the shared memory, so that the media data stream stored in the shared memory can be quickly read and sent to an associated second terminal which is accessed after, when the first terminal and the second terminal both need to acquire the same media data stream, the number of processes for directly acquiring the data stream is reduced, and the media data stream is directly read from the shared memory through the distribution process for distribution, thereby improving the processing efficiency of the media data stream.
The application scenario also provides an application scenario, and the application scenario applies the data stream processing method. Specifically, the application of the data stream processing method in the application scene is as follows:
as shown in fig. 8, in the video conference system, the media gateway mainly realizes interconnection and interworking between the video conference terminal and different communication networks, and in particular, interconnection and interworking between the media gateway and a cloud conference application side, such as a conference app media side, can be realized through a transmission network. The cloud conference is an efficient, convenient and low-cost conference form based on a cloud computing technology. The user can rapidly and efficiently share voice, data files and videos with all groups and clients in the world synchronously by simply and easily operating through an internet interface, and the user is helped by a cloud conference service provider to operate through complex technologies such as data transmission, processing and the like in the conference. At present, domestic cloud conference mainly focuses on service contents mainly in a SaaS (Software as a Service ) mode, including service forms of telephone, network, video and the like, and video conference based on cloud computing is called as a cloud conference. In the cloud conference era, the transmission, processing and storage of data are all processed by the computer resources of video conference factories, and users can carry out efficient remote conferences without purchasing expensive hardware and installing complicated software. The cloud conference system supports the dynamic cluster deployment of multiple servers, provides multiple high-performance servers, and greatly improves conference stability, safety and usability. In recent years, video conferences are popular for a plurality of users because of greatly improving communication efficiency, continuously reducing communication cost and bringing about upgrade of internal management level, and have been widely used in various fields of transportation, operators, education, enterprises and the like. Undoubtedly, the video conference has stronger attraction in convenience, rapidness and usability after the cloud computing is applied, and the video conference application is required to be stimulated.
The media gateway may be a multi-process architecture server, and for video terminal 1 to video terminal N, the media gateway initiates a media process associated with each video terminal, such as media process 1 associated with video terminal 1, and media process N associated with video terminal N. Each Media Process MP (Media Process) represents a Media processing Process, and each video terminal corresponds to a Media Process MP Process. The media process MP is used for receiving the uplink audio and video stream from the video terminal and sending the uplink audio and video stream to the conference app end through the transmission network; and receiving the audio and video streams from the conference app through a transmission network, and transmitting the audio and video streams to the video terminal after transcoding and converging processing. If multiple video terminals join the same conference and access the same media gateway, and other apps may participate in the conference, multiple media processes MP on the media gateway need to repeatedly pull the audio/video stream from the conference app from the transmission network and transcode, which may cause unnecessary performance overhead. In order to save system overhead, in this embodiment, one of the media processes MP pulls the audio and video stream from the conference app, and codes the audio and video stream, and other media processes MP requiring the audio and video stream can directly read from the shared memory.
Specifically, the conference media gateway is configured to receive an uplink audio and video stream of the video terminal, and send a downlink audio and video stream to the video terminal, where the downlink audio and video stream is a multi-process model, and each process corresponds to one path of video terminal. For the situation that a plurality of video terminals join the same conference and access the same media gateway, each process of the media gateway can repeatedly pull one downlink audio/video stream. Shared memory is a common way of inter-process communication. Common shared memory modes include write-read, write-read-many, and the like. One write-read refers to one process writing to the shared memory, and one process consumes from the shared memory, namely, a single producer mode and a single consumer mode, so that no lock can be realized; multiple writing and multiple reading refer to that multiple processes write into the shared memory, and multiple processes read the shared memory, so that locking processing is needed. Typically, a piece of written data can only be consumed by one process, and each process moves the pointer of the response after consuming. In this embodiment, a write-many-read scenario is involved, and each piece of written data is consumed by all the reading processes, i.e. each reading process consumes the full amount of data in the shared memory.
Specifically, as shown in fig. 9, when writing an audio/video stream acquired from a transmission network in a shared memory through a designated multiplexing process, the shared memory of the second level is written first, and then the shared memory of the first level is written. Specifically, step 901, determining a shared memory, which may be specifically determined by a server, specifically by a media gateway in a conference system through a multiplexing process; step 902, carrying out stream pulling and transcoding processing, specifically carrying out stream pulling and transcoding processing from a process in a transmission network by a multiplexing process, and obtaining an audio and video stream. Step 903, determining the latest writing position in the second-level shared memory, specifically determining the latest writing position pos in the second-level shared memory by the multiplexing process; step 904, obtaining the writing authority of the latest writing position, and specifically obtaining the writing authority of the latest writing position pos through a CAS algorithm; step 905, writing the code stream and the code stream information into a designated shared memory location, and writing the obtained code stream and the code stream information, including the check information of the code stream, into the designated shared memory location in the second-stage shared memory by the multiplexing process; step 906, writing the code stream information into the first-stage shared memory, specifically, writing the code stream information into the first-stage shared memory by a multiplexing process; in step 907, the latest writing position of the first-stage shared memory is updated, so that the guided multiplexing process writes the code stream obtained from the transmission network into the shared memory of the two-stage structure.
Further, as shown in fig. 10, when the code stream is read from the shared memory through the designated distribution process, in step 1001, the shared memory is determined, specifically, the shared memory may be determined by the media gateway through the distribution process; step 1002, determining the latest writing position of the first-level shared memory, which may be specifically determined by the media gateway through a distribution process; step 1003, reading the code stream information of the node corresponding to the latest writing position, which specifically may be that the distribution process reads the code stream information of the storage node corresponding to the latest writing position of the first-stage shared memory; step 1004, checking the read code stream information; step 1005, judging whether the verification is passed; if the check is not passed, step 1006 is executed to discard the read code stream information, i.e., discard the read code stream information when the check is not passed, and return to the read process again. Step 1007, if the verification is passed, the code stream is read from the second level shared memory, specifically, the code stream can be read from the second level shared memory based on the code stream information through a distribution process; step 1008, checking the read code stream; step 1009, judging whether the verification is passed; if the check is not passed, executing step 1010, discarding the read code stream, and returning to the read process again; if the verification is passed, step 1011 is executed to send the read code stream to the terminal associated with the distribution process, and the next reading is performed.
In this embodiment, the same media gateway may be provided with a plurality of shared memory queues, where the shared memory is divided into two levels, where the first level shared memory is used to record meta information such as an address of a code stream stored in the shared memory, and the second level shared memory is used to store an actual data frame of the code stream. Therefore, the first-stage shared memory occupies a small space and can be used in a plurality of ways, and the first-stage shared memory can also share the storage space of the second-stage shared memory to reduce the shared memory space because the first-stage shared memory cannot be used at the same time in practice. Moreover, for audio and video streams, the data stored in the second level shared memory, i.e., the element sizes in the second level shared memory, may be different, while the element structures of the first level shared memory may be the same.
For the first level shared memory, the structure may be as follows,
pos elem_size elem_count elem elem …… …… ……
the shared memory is in a ring array structure, pos, elem_size, elem_count and the like are metadata, wherein the metadata respectively occupy 4 bytes, and the metadata is followed by a fixed-length data node elem. pos represents the latest writing position in the first-stage shared memory, pos is a monotonically increasing number, and the actual position in the array is pos% elem_count, i.e. the remainder result of the latest writing position and the number of elements. Common shared memory queues have head and tail, respectively representing the head and tail of the queue, which moves when writing data and moves when reading data. One elem is read by a plurality of processes, and the queue head cannot be moved after the reading is finished, so that a pos is used to represent the latest writing position. elem_size is the size of each element in the first level shared memory, elem_count is the number of elements in the first level shared memory, and elem is a specific element in the first level shared memory. The structure for each elem is as follows,
seq timestamp data_size data_ptr checksum1
Wherein seq is the frame number of the code stream, timestamp is the time stamp when writing, data_size is the size of each data frame in the code stream, data_ptr is the pointer stored by the actual frame, and checksum1 is the checksum generated by seq, timestamp, data _size and data_ptr.
Specifically, for the first-level shared memory, only one writing process exists, no competition exists during writing, and locking processing is not needed. When the process reads data for the first time, it can acquire the latest frame according to pos and periodically read new data frame. If cur_pos currently consumed by the process is smaller than pos, the data can be continuously consumed from the shared memory. The shared memory needs to be able to buffer data with enough duration, but the writing space cannot be guaranteed not to be read at the same time during writing, that is, the situation that the writing coverage is performed after the reading is completed exists, and at this time, whether the read data is valid complete data can be guaranteed through a checksum 1. Specifically, after the data are read, detecting whether the checksum1 is legal or not, and if so, acquiring an actual data frame according to the data_ptr; if not, it is indicated that the reading process has been delayed for a longer time, and to ensure real-time processing of the data stream, the data stream read may be discarded and the next element elem accessed. In addition, if the latest writing position pos-current reading position cur_pos > element number elem_size of the first-stage shared memory indicates that the process is too late, the process can be read from the latest beginning, so as to ensure the real-time performance of data stream processing.
For the second level shared memory, the structure may be as follows,
pos elem_size elem_count frame_elem frame_elem …… …… ……
the pos represents the latest writing position in the second-level shared memory, elem_size is the size of each element in the second-level shared memory, elem_count is the number of elements in the second-level shared memory, frame_elem is a specific element in the second-level shared memory, and each frame_elem is fixed in length. The structure of frame elem is as follows,
write_id seq timestamp data_size data checksum1 checksum2
wherein data_ptr in the first level shared memory element points to frame_element in the second level shared memory. The data is specifically stored code stream data, the data part is designed according to a 1080P frame size space, and the size of a naked stream with a 1080P color format of YUV of one frame can be 1920x1080x1.5= 3110400.write_id represents the writer identification, e.g., may be the memory start address representation of the first level shared memory queue, seq is the frame number of the code stream, timestamp is the timestamp of writing, data_size is the size of each data frame in the code stream, checksum1 is the checksum generated by seq, timestamp, data _size and data_ptr, and checksum2 may be the checksum generated by each item in frame_elem.
The second level shared memory may have multiple processes writing, and the CAS algorithm is used to ensure that the process with the write authority performs the write operation on the corresponding frame_elem. Similarly, pos in the second level shared memory is also incremental, avoiding the ABA problem of CAS algorithm. Further, the write-over-read situation is further processed, when the data is read, the read result can be checked seq, timestamp, checksum, the read result needs to be consistent with the first shared memory structure, the write_id needs to be checked, meanwhile, the checksum2 is checked, when the checksum2 is written into the frame element, the checksum generated by all data in the frame element is checked, if the checksum does not pass, the data is damaged, the write-over situation exists, and the discard processing can be performed.
In this embodiment, when the plurality of terminals respectively acquire the code streams through the respective processes, one process is selected to perform the pulling, the pulled code stream is written into the two-stage shared memory, and other processes directly read the code stream from the shared memory, so that the one-write-multiple-read pulling multiplexing is realized based on the shared memory, the repeated pulling processing of the code stream by all the processes is avoided, and the processing efficiency of the code stream is improved.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a data stream processing device for implementing the above related data stream processing method. The implementation of the solution provided by the apparatus is similar to the implementation described in the above method, so the specific limitation of the embodiment of one or more data stream processing apparatus provided below may refer to the limitation of the data stream processing method described above, and will not be repeated here.
In one embodiment, as shown in fig. 11, there is provided a data stream processing apparatus 1100, comprising: a data stream acquisition module 1102, a data stream writing module 1104, a stored information writing module 1106, a data stream reading module 1108, and a data stream distribution module 1110, wherein:
a data stream obtaining module 1102, configured to obtain a target data stream to be distributed through a multiplexing process;
a data stream writing module 1104, configured to write the target data stream into a data stream storage area of the shared memory through a multiplexing process, and obtain storage information of the target data stream;
the storage information writing module 1106 is configured to write storage information into a data stream information area of the shared memory through the multiplexing process, and send a target data stream to a terminal associated with the multiplexing process;
A data stream reading module 1108, configured to read the storage information from the data stream information area through at least one distribution process, and obtain the target data stream from the data stream storage area according to the read storage information;
the data stream distribution module 1110 is configured to send, through at least one distribution process, a target data stream obtained from the shared memory to a terminal associated with the distribution process.
In one embodiment, the data stream writing module 1104 includes a data stream storage area determination module, a target data stream writing module, and a storage information acquisition module; wherein: the data stream storage area determining module is used for determining a preset data stream storage area of the shared memory through the multiplexing process; a target data stream writing module for writing a target data stream in a data stream storage location in the data stream storage area; and the storage information acquisition module is used for acquiring the storage information of the target data stream according to the data stream storage position.
In one embodiment, the target data stream writing module is further configured to determine a data stream storage location in the data stream storage area, and attribute information of the target data stream; writing the target data stream and attribute information in the data stream storage position; the storage information acquisition module is further used for obtaining the storage information of the target data stream according to the data stream storage position and the attribute information.
In one embodiment, the target data stream writing module is further configured to determine a data stream storage location in the data stream storage area according to the data stream storage location identifier of the data stream storage area; acquiring a write permission for a data stream storage position; based on the write rights, the target data stream is written in the data stream storage location.
In one embodiment, the stored information writing module 1106 includes a data stream information area determination module, an information storage location determination module, and an information storage location writing module; wherein: the data stream information area determining module is used for determining the data stream information area of the shared memory through the multiplexing process; the information storage position determining module is used for determining the information storage position in the data stream information area according to the information storage position identification of the data stream information area; and the information storage position writing module is used for writing the storage information in the information storage position.
In one embodiment, the data stream reading module 1108 includes a distribution process information area determination module, a storage information reading module, and a target data stream reading module; wherein: the distribution process information area determining module is used for determining a data flow information area of the shared memory through at least one distribution process; the storage information reading module is used for reading the storage information from the data stream information area according to the information storage position identification of the data stream information area; and the target data stream reading module is used for obtaining the target data stream from the data stream storage area of the shared memory according to the read storage information.
In one embodiment, the target data stream reading module comprises a stored information checking module, a stored information processing module and a target data stream obtaining module; wherein: the storage information verification module is used for carrying out storage information verification on the read storage information to obtain a storage information verification result; the storage information processing module is used for reading the data stream from the data stream storage area of the shared memory according to the storage information when the verification result of the storage information is that the verification is passed; and the target data stream obtaining module is used for obtaining the target data stream according to the reading result when the reading result obtained by reading the data stream passes the data stream verification.
In one embodiment, the storage information verification module is further configured to extract the storage information to be verified and the first verification reference information from the read storage information; generating storage matching information according to storage information verification conditions and storage information to be verified; and checking and matching the stored matching information with the first checking reference information to obtain a stored information checking result.
In one embodiment, the target data stream obtaining module is further configured to determine data stream information to be checked, first check reference information and second check reference information according to a reading result obtained by reading the data stream; according to the data stream information to be checked, respectively generating storage matching information and data stream matching information according to storage information checking conditions and data stream checking conditions; respectively checking and matching the stored matching information with the first checking reference information, and checking and matching the data stream matching information with the second checking reference information to obtain a data stream checking result; and when the data stream verification result is that the verification passes, obtaining a target data stream according to the reading result.
In one embodiment, the storage information reading module is further configured to obtain an information storage location identifier of the data stream information area; determining a current position identifier of an information storage position read currently by a distribution process; determining a target position identifier from the information storage position identifiers of the data stream information area according to the current position identifier; the stored information is read from the data stream information area based on the destination location identification.
In one embodiment, the target data stream is a media data stream; the data stream obtaining module 1102 is further configured to obtain, when the first terminal accesses the media network, a media data stream from the media network through a multiplexing process associated with the first terminal; the storage information writing module 1106 is further configured to send the media data stream to the first terminal through a multiplexing process; the data stream reading module 1108 is further configured to, when at least one second terminal accesses the media network, read the storage information from the data stream information area through a distribution process associated with the second terminal, and obtain the media data stream from the data stream storage area according to the read storage information; the data stream distribution module 1110 is further configured to send, through a distribution process, a media data stream obtained from the shared memory to the second terminal.
The respective modules in the above-described data stream processing apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 12. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing target data stream data. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data stream processing method. It will be appreciated by those skilled in the art that the structure shown in fig. 12 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-described method embodiments.
It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description. The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims (15)

1. A method of data stream processing, the method comprising:
acquiring a target data stream to be distributed through a multiplexing process;
writing the target data stream into a data stream storage area of a shared memory through the multiplexing process, and obtaining storage information of the target data stream;
writing the storage information into a data stream information area of the shared memory through the multiplexing process, and sending the target data stream to a terminal associated with the multiplexing process;
Reading the storage information from the data stream information area through at least one distribution process, and obtaining the target data stream from the data stream information area according to the read storage information;
and sending the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
2. The method according to claim 1, wherein writing the target data stream into a data stream storage area of a shared memory by the multiplexing process and obtaining storage information of the target data stream, comprises:
determining a preset data stream storage area of the shared memory through the multiplexing process;
writing the target data stream in a data stream storage location in the data stream storage area;
and obtaining the storage information of the target data stream according to the data stream storage position.
3. The method of claim 2, wherein the writing the target data stream in a data stream storage location in the data stream storage area comprises:
determining a data stream storage location in the data stream storage area and attribute information of the target data stream;
Writing the target data stream and the attribute information in the data stream storage location;
the obtaining the storage information of the target data stream according to the data stream storage position includes:
and obtaining the storage information of the target data stream according to the data stream storage position and the attribute information.
4. The method of claim 2, wherein the writing the target data stream in a data stream storage location in the data stream storage area comprises:
determining a data stream storage position in the data stream storage area according to the data stream storage position identification of the data stream storage area;
acquiring a write permission for the data stream storage location;
writing the target data stream in the data stream storage location based on the write permission.
5. The method of claim 1, wherein writing the storage information into the data stream information area of the shared memory by the multiplexing process comprises:
determining a data flow information area of the shared memory through the multiplexing process;
determining an information storage position in the data stream information area according to the information storage position identification of the data stream information area;
Writing the stored information in the information storage location.
6. The method according to claim 1, wherein said reading said stored information from said data stream information area by at least one distribution process and obtaining said target data stream from said data stream storage area based on said read stored information, comprises:
determining a data flow information area of the shared memory through at least one distribution process;
reading the storage information from the data stream information area according to the information storage position identification of the data stream information area;
and obtaining the target data stream from the data stream storage area of the shared memory according to the read storage information.
7. The method of claim 6, wherein the obtaining the target data stream from the data stream storage area of the shared memory based on the read storage information comprises:
performing storage information verification on the read storage information to obtain a storage information verification result;
when the stored information verification result is that verification passes, data stream reading is carried out from the data stream storage area of the shared memory according to the stored information;
And when the reading result obtained by reading the data stream passes the data stream verification, obtaining the target data stream according to the reading result.
8. The method according to claim 7, wherein the performing the stored information verification on the read stored information to obtain a stored information verification result includes:
extracting storage information to be checked and first check reference information from the read storage information;
generating storage matching information according to the storage information to be checked and storage information checking conditions;
and carrying out verification matching on the storage matching information and the first verification reference information to obtain a storage information verification result.
9. The method of claim 7, wherein when the read result of the data stream read passes a data stream check, obtaining the target data stream according to the read result comprises:
determining data flow information to be checked, first check reference information and second check reference information according to a reading result obtained by reading the data flow;
according to the data stream information to be checked, respectively generating storage matching information and data stream matching information according to storage information checking conditions and data stream checking conditions;
Respectively carrying out check matching on the storage matching information and the first check reference information, and carrying out check matching on the data stream matching information and the second check reference information to obtain a data stream check result;
and when the data stream verification result is that the verification is passed, obtaining the target data stream according to the reading result.
10. The method of claim 6, wherein said reading said stored information from said data stream information area based on an information storage location identification of said data stream information area comprises:
acquiring an information storage position identifier of the data stream information area;
determining a current position identifier of an information storage position currently read by the distribution process;
determining a target position identifier from the information storage position identifiers of the data stream information area according to the current position identifier;
and reading the storage information from the data stream information area based on the target position identification.
11. The method according to any one of claims 1 to 10, wherein the target data stream is a media data stream; the obtaining, by the multiplexing process, the target data stream to be distributed includes:
When a first terminal accesses a media network, acquiring a media data stream from the media network through a multiplexing process associated with the first terminal;
the sending the target data stream to the terminal associated with the multiplexing process includes:
transmitting the media data stream to the first terminal through the multiplexing process;
the reading the storage information from the data stream information area through at least one distribution process, and obtaining the target data stream from the data stream information area according to the read storage information, including:
when at least one second terminal accesses the media network, reading the storage information from the data stream information area through a distribution process associated with the second terminal, and obtaining the media data stream from the data stream storage area according to the read storage information;
the sending, by at least one of the distribution processes, the target data stream obtained from the shared memory to a terminal associated with the distribution process, including:
and sending the media data stream obtained from the shared memory to the second terminal through the distribution process.
12. A data stream processing apparatus, the apparatus comprising:
the data stream acquisition module is used for acquiring a target data stream to be distributed through a multiplexing process;
the data stream writing module is used for writing the target data stream into a data stream storage area of the shared memory through the multiplexing process and obtaining storage information of the target data stream;
the storage information writing module is used for writing the storage information into a data stream information area of the shared memory through the multiplexing process and sending the target data stream to a terminal associated with the multiplexing process;
a data stream reading module, configured to read the storage information from the data stream information area through at least one distribution process, and obtain the target data stream from the data stream storage area according to the read storage information;
and the data stream distribution module is used for transmitting the target data stream obtained from the shared memory to a terminal associated with the distribution process through at least one distribution process.
13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 11 when the computer program is executed.
14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.
15. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 11.
CN202210079568.1A 2022-01-24 2022-01-24 Data stream processing method, device, computer equipment and storage medium Pending CN116521385A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210079568.1A CN116521385A (en) 2022-01-24 2022-01-24 Data stream processing method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210079568.1A CN116521385A (en) 2022-01-24 2022-01-24 Data stream processing method, device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116521385A true CN116521385A (en) 2023-08-01

Family

ID=87394567

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210079568.1A Pending CN116521385A (en) 2022-01-24 2022-01-24 Data stream processing method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116521385A (en)

Similar Documents

Publication Publication Date Title
CN107453896B (en) Method and device for processing multiple block chain network data and server
CN111625452B (en) Flow playback method and system
CN110826799B (en) Service prediction method, device, server and readable storage medium
WO2021057084A1 (en) Blockchain-based data processing method and apparatus, system, and electronic device
WO2023147758A1 (en) Method and apparatus for processing cloud game resource data, and computer device and storage medium
CN108829753A (en) A kind of information processing method and device
CN110442424A (en) A kind of method and apparatus for realizing virtual machine management platform dynamic configuration rule
CN110311855B (en) User message processing method and device, electronic equipment and storage medium
CN113687958A (en) Data processing method, system, computer device and storage medium
US20180060974A1 (en) Social networking relationships processing method, system, and storage medium
CN116521385A (en) Data stream processing method, device, computer equipment and storage medium
CN116233253A (en) Service processing method, device, computer equipment and storage medium
CN117435215A (en) Development environment deployment method, system, computer device and storage medium
CN114566173A (en) Audio mixing method, device, equipment and storage medium
CN111400598A (en) Information push method, server, multi-port repeater and storage medium
CN115730016B (en) Data synchronization method, system, device, computer equipment and storage medium
CN112306337B (en) Conference same screen system, method and device, computer equipment and storage medium
CN113553347B (en) Block chain-based data processing method, device, equipment and storage medium
CN112804312B (en) File uploading method, equipment and computer readable medium
CN113495982B (en) Transaction node management method and device, computer equipment and storage medium
CN116760885B (en) Method, device, equipment, medium and program product for managing computing power network business
CN117132348A (en) Resource interaction method, device, computer equipment and storage medium
CN116954474A (en) Method, apparatus, device, storage medium and program product for data reading
CN116366724A (en) Business process processing method, device, computer equipment and storage medium
CN117707602A (en) Data request processing method, device, server and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40091099

Country of ref document: HK