CN114520832A - Data stream processing method, device, equipment and computer storage medium - Google Patents
Data stream processing method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN114520832A CN114520832A CN202011299080.7A CN202011299080A CN114520832A CN 114520832 A CN114520832 A CN 114520832A CN 202011299080 A CN202011299080 A CN 202011299080A CN 114520832 A CN114520832 A CN 114520832A
- Authority
- CN
- China
- Prior art keywords
- data stream
- subscription
- distributed
- subscriber
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/542—Event management; Broadcasting; Multicasting; Notifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The application provides a data stream processing method, a data stream processing device, data stream processing equipment and a computer storage medium, relates to the technical field of computers, and aims to improve the flexibility of a subscriber end requesting a data stream through a data stream processing engine. The method comprises the following steps: a data stream processing engine acquires a subscription broadcast stream of a data stream requested by a subscription end, and updates subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end; and acquiring a data stream to be distributed sent by a data generation end, determining each target subscription end requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscription end. According to the method, the subscription terminal can flexibly request the data stream provided by the data generation terminal only by recording and updating the subscription configuration information, so that the flexibility of the subscription terminal requesting the data stream through the data stream processing engine is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a computer storage medium for processing a data stream.
Background
In streaming data (streaming data) processing, a data stream provided by a data generation end is pushed to a subscription end (i.e. an end requesting the data stream) through a streaming data processing framework provided by a data stream processing engine; however, the data stream provided by the current data generating end directly indicates the information of the subscribing end receiving the data stream, and then the data stream processing engine can only send the data stream to the subscribing end indicated in the data stream, and cannot send the data stream to other subscribing ends except the specified subscribing end; therefore, if other subscribers want to request the data stream, they need to inform the data generation end through tasks, and then the data generation end regenerates the data stream, and sends the regenerated data stream to the other subscribers through the data stream processing engine, so that the delivery flexibility of the data stream is poor, and the flexibility of the subscribers requesting the data stream is low.
Disclosure of Invention
The embodiment of the application provides a data stream processing method, a data stream processing device, a data stream processing apparatus and a computer storage medium, which are used for improving the flexibility of a subscriber end requesting a data stream through a data stream processing engine.
In a first aspect of the present application, a data stream processing method is provided, which is applied to a data stream processing engine, and includes:
acquiring a subscription broadcast stream of a data stream requested by a subscriber, and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber;
acquiring a data stream to be distributed sent by a data generation end, determining each target subscription end requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscription end.
In a second aspect of the present application, there is provided a data stream processing apparatus, including:
the broadcast stream processing unit is used for acquiring a subscription broadcast stream of a data stream requested by a subscription end and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
and the data stream processing unit is used for acquiring the data stream to be distributed sent by the data generation end, determining each target subscription end requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscription end.
In a possible implementation manner, the data stream processing unit is specifically configured to:
and if the data stream to be distributed carries the subscriber indication information, determining each target subscriber in the subscribers indicated by the subscriber indication information based on the subscription configuration information.
In a possible implementation manner, the broadcast stream processing unit is further configured to, before acquiring a subscription broadcast stream of a data stream requested by a subscriber, acquire the subscription broadcast stream from the first message middleware, and acquire, from the second message middleware, a data stream to be distributed, and tag one of the acquired subscription broadcast stream or the data stream to be distributed with a tag to form a data stream to be processed; and
the broadcast stream processing unit is specifically configured to: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the label;
the data stream processing unit is specifically configured to: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the label.
In a third aspect of the present application, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of the first aspect and any one of the possible embodiments when executing the program.
In a fourth aspect of the present application, a computer program product is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various possible implementations of the first aspect described above.
In a fifth aspect of the present application, there is provided a computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of the first aspect and any one of the possible embodiments.
Due to the adoption of the technical scheme, the embodiment of the application has at least the following technical effects:
on the first hand, in the embodiment of the application, the data stream processing engine can dynamically acquire information of a data stream requested by a subscription end based on a subscription broadcast stream, and then the data stream processing engine distributes the data stream to be distributed to each target subscription end based on the recorded subscription configuration information.
In a second aspect, in the embodiment of the present application, the subscription configuration information is updated in real time based on the subscription broadcast stream of the subscriber, so that the accuracy of the determined target subscriber can be improved, and the accuracy of sending the data stream to be distributed is improved.
In the third aspect, in the embodiment of the application, the dynamic subscription of the data stream to be distributed is realized without adding additional module components, so that the system complexity is reduced, and the efficiency of dynamically requesting the data stream by the data stream processing engine at the subscription end is improved.
Drawings
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application;
fig. 2 is an exemplary diagram of another application scenario provided in an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a process of a data stream processing method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a process of sending a data stream to be distributed according to an embodiment of the present application;
fig. 5 is a schematic diagram of a process of determining a target subscriber according to an embodiment of the present application;
fig. 6 is a schematic process diagram of a data stream processing method according to an embodiment of the present application;
fig. 7 is a structural diagram of a data stream processing apparatus according to an embodiment of the present application;
fig. 8 is a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the drawings and specific embodiments.
In order to facilitate those skilled in the art to better understand the technical solutions of the present application, the following description refers to the technical terms of the present application.
1) Data stream processing engine
Data stream processing engines data processing engines that process streaming data (i.e., data streams) may include, but are not limited to, Apache, Flink, Storm, Samza, etc.; the Flink is a framework for performing stateful computation processing on unbounded and bounded data streams, and is a relatively efficient data stream processing engine which considers that all data exists in the form of streams.
2) Message queue
A Message Queue (MQ) is a container for storing messages (data), that is, a Queue for storing data to be transmitted; the message queue is an asynchronous inter-service communication mode, is an important component in a distributed system, can be used for publishing and subscribing messages, mainly solves the problems of application coupling, asynchronous messages, traffic peak clipping and the like, and realizes a high-performance, high-availability, scalable and final consistency architecture; commonly used message queues are rockmq, RabbitMQ, Kafka, etc.; where Kafka is a distributed, multi-partitioned, multi-replica, multi-subscriber, zookeeper-based coordinated message queue (also known as a distributed log system).
3) Subscribing terminal and data generating terminal
The subscription end in the embodiment of the present application is a device or a service that requests a data stream (streaming data), and the subscription end may be a terminal device, or may be a message queue, or a service platform; the data generation end in the embodiment of the present application is a device or a service that provides or generates a data stream, and the data generation end may be, but is not limited to, a message queue, a service platform, and the like; the service platform related in the embodiment of the present application is also referred to as a system service, and may include, but is not limited to, a game service platform, a video service platform, a live broadcast service platform, a content sharing service platform, a traffic service platform, and the like.
4) Subscribing to broadcast streams
The subscription broadcast stream in the embodiment of the present application is a broadcast message for requesting a data stream, and the subscription broadcast stream may be triggered by a subscriber, and may carry subscriber indication information indicating an identity of the subscriber, and may also carry data stream indication information indicating a data stream requested by the subscriber.
5) Data stream, data stream to be distributed, data stream to be processed
The data stream in the embodiment of the application is streaming data provided by a data generation end; the data stream to be distributed is the data stream which is sent to a data stream processing engine by a data generation end and is to be processed by the data stream processing engine; the data stream to be processed is formed by labeling one of the acquired subscription broadcast stream or the data stream to be distributed with a tag by a data stream processing engine.
6) Subscription configuration information
The subscription configuration information in the embodiment of the present application is information used for indicating a subscriber to request a data stream, and the subscription configuration information may include subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber.
7) Cloud service technology (Cloud technology)
The cloud service technology is a general name of a network technology, an information technology, an integration technology, a management platform technology, an application technology and the like based on cloud computing business model application, can form a resource pool, is used as required, and is flexible and convenient. Cloud service technology is an important support; background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data of different levels are processed separately, and various industrial data need strong system background support and can only be realized through a cloud service technology.
The following explains the concept of the present application.
In streaming data (streaming data) processing, a streaming data processing framework of a data stream processing engine is generally used to push a data stream sent by a data generation end to a fixed subscription end; in the process of processing the streaming data, after a subscription end subscribes a data stream of a data generation end, a streaming data processing framework indicates a fixed subscription end subscribing the data stream in a task; after a data generation end starts a task, determining a fixed subscription end for receiving a data stream based on the task, indicating the fixed subscription end in the data stream generated after the task is started, and not modifying information of the fixed subscription end for receiving the data stream after the data generation end starts a processing task; after receiving the data stream, the stream processing engine only supports the indication based on the data stream, and pushes the received data stream to the fixed subscriber terminal indicated by the data stream, but cannot deliver the data stream to other subscriber terminals except the fixed subscriber terminal, so that each subscriber terminal cannot flexibly subscribe the data stream provided by the data generating terminal.
In view of the above, the inventors devised a data stream processing method, apparatus, device, and computer storage medium; considering that a fixed subscriber for receiving a data stream is already specified in the data stream in the related art, and therefore a data stream processing engine can only push the data stream to the fixed subscriber, in the embodiment of the present application, it is designed that the data stream processing engine does not indicate the subscriber, and sends the data stream to the subscriber requesting the data stream according to subscription configuration information of the subscriber in real time; meanwhile, in order to improve the accuracy of data stream distribution, the data stream processing engine updates the subscription configuration information in real time based on the subscription broadcast stream of the subscriber terminal in the process of processing the data stream, and then distributes the received data stream to the subscriber terminal requesting the data stream based on the updated subscription configuration information.
In order to more clearly understand the design idea of the present application, an application scenario of the embodiment of the present application is described below as an example.
Referring to fig. 1, a diagram of an exemplary application scenario for subscribing to a data stream is shown. The application scenario includes at least one data generation end 100, a data stream processing engine 200, and at least one subscriber end 300, where the data generation end 100 and the data stream processing engine 200 may communicate with each other, and the data stream processing engine 200 and the subscriber end 300 communicate with each other; wherein:
the data generating end 100 may generate a data stream after starting the task, and send the data stream to the data stream processing engine 200.
The data stream processing engine 200 (such as but not limited to include 200-1, 200-2, or 200-3 illustrated in the figure) may acquire a subscription broadcast stream of the data stream requested by the subscriber 300, and update the subscription configuration information based on the subscription broadcast stream; acquiring a data stream to be distributed sent by the data generation end 100, and sending the data stream to be distributed to each target subscription end requesting the data stream to be distributed based on the subscription configuration information; the data stream to be distributed is a data stream generated by the data generating end 100.
The subscriber 300 may send a subscription broadcast stream requesting a data stream to the data stream processing engine 200 and receive a data stream to be distributed sent by the data stream processing engine 200.
As an embodiment, in consideration of the network environment and the influence of the storage capacity and the processing capacity of the data stream processing engine 200, if the number of data streams and subscription broadcast streams reaching the data stream processing engine 200 in a short time is too large, accumulation or blocking of the data streams and subscription broadcast streams may occur, so as to have a serious influence on the whole data stream processing system, so in the embodiment of the present application, the data streams and subscription broadcast streams are designed to be cached by using the message middleware, so as to avoid adverse influence of a large amount of streaming data on the data stream processing engine 200.
Specifically, referring to fig. 2, another exemplary diagram of an application scenario for subscribing to a data stream is provided. The application scenario includes at least one data generation end 100, a data stream processing engine 200 and at least one subscription end 300, wherein a first message middleware 210 is arranged between the data stream processing engine 200 and the subscription end 300, and a second message middleware 220 is arranged between the data generation end 100 and the data stream processing engine 200; wherein:
the data generating end 100 (such as but not limited to including the generating end 100-1, 100-2 or 100-3 illustrated in the figure) may generate a data stream after starting the task, and send the data stream to the second message middleware 220.
The subscriber 300 (such as may include, but is not limited to, 300-1 or 300-2 illustrated in the figures) may send the subscription broadcast stream directly to the data stream processing engine 200; the subscriber 300 may also carry the subscriber indication information and the data stream indication information in a subscription request and send the subscription request to the service node 400, and the service node 400 processes the subscription request into a subscription broadcast stream and sends the subscription broadcast stream to the data stream processing engine 200 through the first message middleware 210; the service node 400 may be an independent server, or may be a server cluster.
After triggering the processing of the data stream and the subscription broadcast stream, the data stream processing engine 200 (such as but not limited to 200-1, 200-2, or 200-3 illustrated in the figure) may directly receive the subscription broadcast stream sent by the subscriber 300, or may obtain the subscription broadcast stream from the first message middleware 210, and update the subscription configuration information based on the subscription broadcast stream; and acquiring the data stream to be distributed from the second message middleware 220, and sending the data stream to be distributed to the subscriber 300 based on the subscription configuration information.
Further, the data stream processing engine 200 may obtain a plurality of subscription broadcast streams from the first message middleware 210 at a time, and store the obtained plurality of subscription broadcast streams and the plurality of data streams to be distributed to its own cache space when obtaining the plurality of data streams to be distributed from the second message middleware 220 at a time; the data stream processing engine 200 may sequentially extract the subscription broadcast stream and the data stream to be distributed from its own cache space for processing; when the data in the buffer space of the device is empty, the subscription broadcast stream is obtained from the first message middleware 210 again, and the data stream to be distributed is obtained from the second message middleware 220.
Further, when the data stream processing engine 200 sequentially takes out the subscribed broadcast streams and the data streams to be distributed from its own cache space for processing, one subscribed broadcast stream or one data stream to be distributed may be taken out from the cache space at a time for processing, or one subscribed broadcast stream and one data stream to be distributed may be taken out from the cache space at a time, and the one subscribed broadcast stream and the one data stream to be distributed that are taken out at the same time constitute one data stream to be processed, and then the data stream processing engine 200 processes the data stream to be processed.
The data processing engine 200 in this embodiment of the application may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, for example, may be a plurality of blockchain servers providing blockchain services in a blockchain technology, or may be a plurality of cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms in a cloud service technology (for example, but not limited to, the cloud servers may include a server 200-1, a server 200-2, or a server 200-3 illustrated in the drawing); the functions of the data processing engine 200 may be implemented by one or more cloud servers, one or more cloud server clusters, and the like.
The data generating end 100 in the embodiment of the present application may be, but is not limited to, a message queue, a service platform, and the like, and the subscriber end 300 may be, but is not limited to, a terminal device, a message queue, a service platform, and the like.
The message middleware (such as the first message middleware 210 or the second message middleware 220) in the embodiment of the application may be, but is not limited to, a message queue, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a plurality of cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud storage, network services, cloud communication, middleware services, platforms, and the like in a cloud service technology; the functions of the message middleware can be realized by one or more cloud servers, one or more cloud server clusters, and the like.
The message queue and the service platform related in the embodiment of the application can be independent physical servers, can also be a server cluster or a distributed system formed by a plurality of physical servers, and can also be a plurality of cloud servers which provide basic cloud computing services such as cloud service, cloud databases, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (content delivery network) and big data and artificial intelligence platforms in the cloud service technology; the functions of the message queue or the service platform may be implemented by one or more cloud servers, may also be implemented by one or more cloud server clusters, and the like.
The terminal devices involved in the embodiments of the present application may be, but are not limited to, mobile terminals, fixed terminals, or portable terminals, such as mobile handsets, stations, units, devices, multimedia computers, multimedia tablets, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, Personal Communication Systems (PCS) devices, personal navigation devices, Personal Digital Assistants (PDAs), audio/video players, digital cameras/camcorders, positioning devices, television receivers, radio broadcast receivers, electronic book devices, game devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof.
Based on the application scenarios of fig. 1 and fig. 2, the following describes an example of a data stream processing method involved in the embodiment of the present application; referring to fig. 3, a data stream processing method applied to the data stream processing engine 200 is provided, which specifically includes the following steps:
step S301, obtaining a subscription broadcast stream of a data stream requested by a subscriber, and updating subscription configuration information based on the obtained subscription broadcast stream, where the subscription broadcast stream carries the subscriber indication information of the subscriber and the data stream indication information of the data stream requested by the subscriber.
Specifically, the obtained subscribed broadcast stream is the subscribed broadcast stream to be processed by the data stream processing engine 200; the data stream processing engine 200 may, but is not limited to, determine subscription configuration information indicated by the subscriber indication information based on the subscriber indication information in the subscription broadcast stream, and then update the determined subscription configuration information based on the total data stream indication information of the subscription broadcast stream.
The subscriber end indication information in the embodiment of the application is used for indicating identity information of the subscriber end, the subscriber end indication information may be but is not limited to identification information of the subscriber end, and the identification information of the subscriber end may be account information of the subscriber end or an Internet Protocol (IP) address of the subscriber end; when the subscriber is a service platform (such as but not limited to a website), the subscriber indication information may also be a website or a domain name of the website, and those skilled in the art may set the subscriber indication information according to actual requirements.
The data flow indication information in the embodiment of the application is used to indicate a data flow requested by a subscriber, and the data flow indication information may be, but is not limited to, a content subject of the data flow, a generation end identifier of a data generation end that generates the data flow, a data type of the data flow, a website or a domain name of a service platform that provides the data flow, and the like.
Step S302, acquiring a data stream to be distributed sent by a data generating end, determining each target subscriber requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscriber.
Specifically, in the data stream to be distributed in the embodiment of the present application, a subscriber that receives the data stream to be distributed may not be specified, that is, the data stream to be distributed does not need to carry subscriber indication information of the subscriber, and further in step S302, the data stream processing engine 200 may directly distribute the data stream to be distributed to each determined target subscriber.
As an embodiment, since the present application embodiment may include multiple subscribers such as a terminal device, a message queue, or a service platform, and different subscribers may request different types of data streams, and if the subscribers are different message queues, different message queues and data transmission protocols between the data stream processing engine 200 are different, in this embodiment, to improve the sending success rate of the data stream to be distributed, the data stream to be distributed may also be sent to each target subscriber according to a data transmission protocol matched with the subscription type of each target subscriber based on the subscription type of the target subscriber, and specific content will be described in detail below.
The following describes the process of the data stream processing engine 200 acquiring the subscription broadcast stream in step S301 and acquiring the data stream to be distributed in step S302:
as an embodiment, in step S301, the data stream processing engine 200 may determine the subscription broadcast stream sent by the subscriber 300 or the first message middleware 210 as the acquired subscription broadcast stream; the subscription broadcast stream may also be actively captured from the first message middleware 210, and the actively captured subscription broadcast stream is determined as the obtained subscription broadcast stream; the subscription broadcast stream captured from the cache space of the subscription broadcast stream can be determined as the acquired subscription broadcast stream; for the application scenario illustrated in fig. 1, the subscriber 300 may directly send a subscription broadcast stream to the data stream processing engine 200, and then the data stream processing engine 200 may determine the subscription broadcast stream sent by the subscriber as the obtained subscription broadcast stream; for the application scenario illustrated in fig. 2, the first message middleware 210 caches subscription broadcast streams of one or more subscribers, and the first message middleware 210 may actively send the subscription broadcast streams to the data stream processing engine 200, or the data stream processing engine 200 actively captures the subscription broadcast streams from the first message middleware 210.
In step S302, the data stream processing engine 200 may determine the data stream sent by the data generation end 100 or the second message middleware 220 as the acquired data stream to be distributed; the data stream can also be actively captured from the second message middleware 220, and the actively captured data stream is determined as the data stream to be distributed; the data stream captured from the self cache space can be determined as the data stream to be distributed; for the application scenario illustrated in fig. 1, the data generation end 100 may directly send a data stream to the data stream processing engine 200, and then the data stream processing engine 200 may determine the data stream sent by the data generation end 100 as a data stream to be distributed; for the application scenario illustrated in fig. 2, one or more data streams of the data generating end 100 are cached in the second message middleware 220, and the second message middleware 220 may actively send the data stream to the data stream processing engine 200, or the data stream processing engine 200 actively captures the data stream from the second message middleware 220 as a data stream to be distributed.
As an embodiment, in order to improve the processing efficiency of the data stream processing engine and improve the accuracy of sending the data stream to be distributed, in this embodiment of the present application, before the data stream processing engine 200 acquires the subscription broadcast stream of the data stream requested by the subscriber in step S301, the data stream processing engine 200 may also acquire the subscription broadcast stream and the data stream to be distributed at the same time, and tag one of the acquired subscription broadcast stream or the data stream to be distributed to form the data stream to be processed; further, when a subscription broadcast stream of a data stream requested by a subscriber is acquired in step S301, acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the tag; when the data stream to be distributed sent by the data generating end is obtained in step S302, the data stream to be distributed may be separated from the data stream to be processed according to the tag;
in the process of composing the data stream to be processed, the data stream processing engine 200 may obtain the subscription broadcast stream from the first message middleware 210 and obtain the data stream to be distributed from the second message middleware 220 at the same time, or the data stream processing engine 200 may obtain the subscription broadcast stream or the data stream to be distributed from the buffer space at the same time; and then the label can be used as a connection character, and the data stream to be processed is obtained after the subscription broadcast stream and the data stream to be distributed are directly spliced.
As an embodiment, in the embodiment of the present application, the data stream processing engine 200 records subscription configuration information of each subscriber, that is, a subscription configuration information set is recorded in the data stream processing engine 200, where the subscription configuration information set includes subscription configuration information of each subscriber that has sent a subscription broadcast stream; when the data stream processing engine 200 is an independent physical server, the subscription configuration information set may be stored locally in the physical server; when the data stream processing engine 200 is a server cluster or a distributed system, it may be enough to record a subscription configuration information set in the server cluster or the distributed system, each server in the server cluster or the distributed system may access the recorded subscription configuration information set, and may update the subscription configuration information in the recorded subscription configuration information set in step S301 in real time according to the acquired subscription broadcast stream.
The data stream processing engine 200 may, but is not limited to, record subscription configuration information of each subscriber in a subscription configuration information set in the form of a key value pair or a fixed data type, for example, when the data stream processing engine 200 is a Flink, the Flink may record subscription configuration information of each subscriber in the form of map < state, cnf > in a Flink state, where the state includes subscriber indication information of a subscriber, and the cnf may, but is not limited to, include data stream indication information.
The following content of the embodiment of the present application describes in detail the process of sending the data stream to be distributed to each target subscriber based on the subscription type of the target subscriber in step S302:
as an embodiment, the matching relationship between the subscription type and the data transmission protocol is not limited too much, and a person skilled in the art may set the matching relationship according to actual requirements, for example, in order to promote the message of data stream processing, the matching relationship between the subscription type and the data transmission protocol may be set in advance, or the matching relationship between the subscription type and the data transmission protocol may be set based on the related information of the subscriber.
As an embodiment, in order to quickly and accurately determine the subscription type of each target subscriber, in this embodiment of the present application, the data stream processing engine 200 may further record subscription type information of each subscriber, where the subscription type information is used to indicate the subscription type of the subscriber; specifically, each subscriber may send respective subscription type information to the data stream processing engine 200 through a message other than the above subscription broadcast stream, and each subscriber may also carry the respective subscription type information in the above subscription broadcast stream and send the information to the data stream processing engine 200; further, after receiving the subscription type information of each subscriber, the data stream processing engine 200 may record the subscription type information in the subscription configuration information, or record the subscription type information in information other than the subscription configuration information; if the data stream processing engine 200 is a Flink, the Flink may record the subscription type information of the subscriber in the cnf in the map < state, cnf >; the Flink may also record the subscription type information of the subscriber in the form of map < key, client >, where the key includes the subscriber indication information of the subscriber, and the client may include, but is not limited to, the subscription type information of the subscriber.
When the subscription broadcast stream may carry the subscription type information of the subscriber, after receiving the subscription broadcast stream, the data stream processing engine 200 may further update the subscription type information of each subscriber based on the subscription type information of the subscriber carried in the subscription broadcast stream; when the data stream to be distributed is sent to each target subscriber, the data stream processing engine 200 may determine the subscription type of each target subscriber based on the recorded subscription type information; and respectively sending the data stream to be distributed to each target subscriber according to the data transmission protocol matched with the subscription type of each target subscriber.
For ease of understanding, the following is an exemplary description of the subscription type of the subscriber involved in the present application:
when the subscription terminal is different message queues, the subscription type may be determined according to the queue types of the message queues, and if the subscription terminal includes ActiveMQ, RabbitMQ, rocktmq, Kafka, and ES, the subscription type may be ActiveMQ type, RabbitMQ type, rocktmq type, Kafka type, and ES type.
When the subscriber includes different service platforms, the subscription type may be, but is not limited to, determined based on the service type of the service platform or the domain name of the corresponding website, and the like.
The subscription end comprises different terminal devices, and the subscription type can be determined based on the type, the network, the position and the like of the terminal device; further, when the subscriber is a terminal device and the data stream processing engine 200 transmit the subscribed broadcast stream and the data stream to be distributed through the first message middleware 210, the data stream processing engine 200 may send the data stream to be distributed to each target subscriber based on, but not limited to, the following manners: respectively determining a message middleware corresponding to the subscription type of the target subscriber terminal aiming at each target subscriber terminal; and respectively sending the data streams to be distributed to the message middleware corresponding to the subscription type of the target subscriber according to a data transmission protocol matched with the subscription type of each target subscriber, so that the message middleware sends the data streams to be distributed to the target subscribers corresponding to each subscription type.
Referring to fig. 4, for example, the determined target subscriber includes a subscriber 1, a subscriber 2, a subscriber 3, and a subscriber 4, where the subscription types of the subscriber 1 to the subscriber 3 are Kafka type subscribers, and when the subscription type of the subscriber 4 is rocktmq, the data stream processing engine 200 determines that the data streams to be distributed bypass the subscriber 1 to the subscriber 3 are in a Kafka type data stream channel when bypassing the data streams (that is, sending the data streams to be distributed to the target subscriber), and sends the data to be distributed to the subscriber 1 to the subscriber 3 according to a Kafka data transmission protocol matched with the Kafka type; meanwhile, the data stream processing engine 200 bypasses the data stream to be distributed of the subscriber terminal 4 to the rockmq type data stream channel, and sends the data to the subscriber terminal 4 according to the rockmq transmission protocol matched with the rockmq type.
As an embodiment, there may be no current subscriber requesting the data stream to be distributed, and in this case, in order to ensure that the task related to the data stream can be performed normally, the data stream to be distributed may be directly discarded, or based on a service requirement, the data stream to be distributed is stored for subsequent processing; specifically, if the target subscriber is not determined based on the subscription configuration information, the data stream to be distributed is discarded.
As an embodiment, in some service scenarios, it may be specified that a data stream generated by a data generation end is expected to be consumed by a subscriber end (that is, the data stream is requested by the subscriber end), and in this case, after it is determined that there is no subscriber end requesting the data stream to be distributed, the data stream to be distributed may be cached to a storage duration, and then, based on an update of subscription configuration information of each subscriber end within the storage duration, it is determined whether to distribute the data stream to be distributed to a corresponding subscriber end;
specifically, referring to fig. 5, if the target subscription end is not determined based on the subscription configuration information, the data stream to be distributed is stored, and when the storage duration of the data stream to be distributed reaches the duration threshold, whether the target subscription end exists is determined again based on the latest subscription configuration information, if the target subscription end exists is determined again, the stored data stream to be distributed is distributed to the re-determined target subscription end, otherwise, whether the number of times of re-determination reaches the set number of times is determined, if the number of times of re-determination does not reach the set number of times, the target subscription end is determined again, the stored data stream to be distributed is discarded, otherwise, the storage duration of the data stream to be distributed is monitored again after the storage duration is cleared, and when the storage duration reaches the duration threshold again, whether the target subscription end exists is determined again based on the latest subscription configuration information, or the storage duration is not cleared, and when the time difference between the storage duration and the latest redetermined time reaches the duration threshold, whether the target subscription end exists is redetermined based on the latest subscription configuration information; the storage duration, the duration threshold and the set times are not limited too much, and can be set by a person skilled in the art according to actual requirements.
As an embodiment, in order to improve the flexibility of delivering a data stream by a data generating end, in the embodiment of the present application, the data generating end 100 may also carry subscription-end indication information in a data stream to be distributed, so as to indicate to the data stream processing engine 200 that only a subscription end corresponding to the subscription-end indication information carried in the data stream to be distributed has an authority to acquire the data stream to be distributed; further, when determining each target subscriber requesting the data stream to be distributed, if the data stream to be distributed carries subscriber indication information, the data stream processing engine 200 determines each target subscriber in the subscribers indicated by the subscriber indication information carried in the data stream to be distributed based on the subscription configuration information.
Specifically, a subscription end requesting the data stream to be distributed may be determined based on the latest subscription configuration information, and then a subscription end indicated by the subscription end indication information carried in the data stream to be distributed is determined as a target subscription end from the determined subscription ends; or determining a subscription end indicated by the subscription end indication information carried in the data stream to be distributed, acquiring the subscription configuration information of the determined subscription end from the latest subscription configuration information, and determining the subscription end requesting the data stream to be distributed in the determined subscription ends as a target subscription end based on the acquired subscription configuration information.
In order to facilitate understanding of the technical solutions provided in the embodiments of the present application, a specific example of a data stream processing method is provided in the following.
In this example, Flink is explained as the data stream processing engine 200.
Referring to fig. 6, in the dynamic subscription process of the Flink-based data stream, the first message middleware aggregates the subscription configuration information of each subscriber into a broadcast (broadcast) subscription stream in a manner of a message queue, and the like, and the second message middleware caches the data stream sent by each data generation end in the second middleware;
the method comprises the steps that a Flink engine acquires a subscription broadcast stream of a subscription end, acquires a data stream sent by a data generation end as a data stream to be distributed, further a task process (task manager) in the Flink executes a task through a task resource (task), and connects the acquired subscription broadcast stream with the data stream to be distributed to obtain data to be processed, wherein the data stream to be processed is actually a spliced new data broadcast stream; the Flink engine can further acquire the data stream to be processed, update and cache the subscription configuration information map < state, cnf > according to the subscription configuration information in the data stream to be processed, and update and cache the client information map < key, client > corresponding to the subscription type; and finally bypassing the data stream to be distributed requested by each target subscriber terminal from the data stream to be processed according to the subscription type, and further sending the data stream to be distributed to each target subscriber terminal subscribing the data stream to be distributed.
In the process of sending the data stream to be distributed, the Flink sends the data stream to be distributed to each target subscriber according to a data transmission protocol matched with the subscription type of each target subscriber; specifically, the Flink may copy and bypass the data streams to be distributed based on the total number of subscription types of the subscription end known in advance, and obtain the data stream copies to be distributed with the total number as side branch data streams; and then determining each target subscriber based on the latest subscription configuration information map < state, cnf >, determining the subscription type (client information) of each target subscriber based on the latest map < key, client >, and then respectively sending each bypass data stream to each corresponding target subscriber according to the data transmission protocol matched with the subscription type of each target subscriber.
According to the data stream processing method and device, only one piece of subscription configuration information of the subscriber is required to be recorded, and the recorded subscription configuration information is updated in real time according to the received subscription broadcast stream, so that the data stream provided by the data generation end can be flexibly requested by the subscriber according to the recorded subscription configuration information, and the flexibility of the data stream requested by the subscriber through the data stream processing engine is improved.
Referring to fig. 7, based on the same inventive concept, an embodiment of the present application provides a data stream processing apparatus 700, including:
a broadcast stream processing unit 701, configured to acquire a subscription broadcast stream of a data stream requested by a subscriber, and update subscription configuration information based on the subscription broadcast stream, where the subscription broadcast stream carries subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber;
a data stream processing unit 702, configured to acquire a data stream to be distributed sent by a data generating end, determine, based on the subscription configuration information, each target subscriber requesting the data stream to be distributed, and send the data stream to be distributed to each target subscriber.
As an example, the apparatus in fig. 7 may be used to implement any of the data stream processing methods discussed above.
As an embodiment, the subscription broadcast stream further carries subscription type information of a subscriber, and the broadcast stream processing unit 701 is further configured to: updating the subscription type information of each subscription terminal based on the subscription type information of the subscription terminal carried in the subscription broadcast stream; and
the data stream processing unit 702 is specifically configured to:
determining the subscription type of each target subscriber based on the recorded subscription type information;
and respectively sending the data stream to be distributed to each target subscriber according to the data transmission protocol matched with the subscription type of each target subscriber.
As an embodiment, the data stream processing unit 702 is further configured to:
if the target subscriber terminal is not determined based on the subscription configuration information, discarding the data stream to be distributed; or storing the data stream to be distributed, re-determining whether a target subscriber exists based on the latest subscription configuration information when the storage duration of the data stream to be distributed reaches a duration threshold, and discarding the stored data stream to be distributed if the target subscriber is not re-determined by the set times.
As an embodiment, the data stream processing unit 702 is specifically configured to:
and if the data stream to be distributed carries the subscriber indication information, determining each target subscriber in the subscribers indicated by the subscriber indication information based on the subscription configuration information.
As an embodiment, the broadcast stream processing unit 701 is further configured to, before acquiring a subscription broadcast stream of a data stream requested by a subscriber, acquire the subscription broadcast stream from the first message middleware, and acquire, from the second message middleware, a tag of the acquired subscription broadcast stream or one of the data streams to be distributed from a data stream to be distributed, and then compose a data stream to be processed; and
the broadcast stream processing unit 701 is specifically configured to: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the label;
the data stream processing unit 702 is specifically configured to: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the label.
An example of the data stream processing apparatus 700 as a hardware entity is a computer device shown in fig. 8, which includes a processor 801, a storage medium 802 and at least one external communication interface 803; the processor 801, the storage medium 802, and the external communication interface 803 are connected by a bus 804.
The storage medium 802 stores therein a computer program;
the processor 801, when executing the computer program, implements a data stream processing method as discussed above.
Fig. 8 illustrates an example of one processor 801, but the number of processors 801 is not limited in practice.
The storage medium 802 may be a volatile storage medium (volatile memory), such as a random-access memory (RAM); the storage medium 802 may also be a non-volatile storage medium (non-volatile memory), such as a read-only memory (rom), a flash memory (flash memory), a hard disk (HDD) or a solid-state drive (SSD), or the storage medium 802 may be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The storage medium 802 may be a combination of the above storage media.
As an embodiment, the processor 801 in fig. 8 may be used to implement the functions of the broadcast stream processing unit 701 and the data stream processing unit 702 in fig. 7.
As an embodiment, each aspect of the video playback control method provided in the embodiments of the present application may also be implemented in the form of a program product including program code for causing a computer device to perform the steps in the data stream processing method according to the various exemplary embodiments of the present application described above in this specification when the program product runs on the computer device.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the above methods of the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
Based on the same technical concept, the embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions that, when executed on a computer, cause the computer to execute the data stream processing method as discussed above.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A data stream processing method applied to a data stream processing engine includes:
acquiring a subscription broadcast stream of a data stream requested by a subscriber, and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber;
acquiring a data stream to be distributed sent by a data generation end, determining each target subscription end requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscription end.
2. The method of claim 1, wherein the subscription broadcast stream further carries subscription type information of a subscriber, and the method further comprises: updating the subscription type information of each subscription terminal based on the subscription type information of the subscription terminal carried in the subscription broadcast stream; and
the sending the data stream to be distributed to each target subscriber includes:
determining the subscription type of each target subscriber based on the recorded subscription type information;
and respectively sending the data stream to be distributed to each target subscriber according to the data transmission protocol matched with the subscription type of each target subscriber.
3. The method according to claim 1 or 2, wherein if no target subscriber is determined based on the subscription configuration information, the method further comprises:
discarding the data stream to be distributed; or
And storing the data stream to be distributed, re-determining whether a target subscription end exists or not based on the latest subscription configuration information when the storage duration of the data stream to be distributed reaches a duration threshold, and discarding the stored data stream to be distributed if the target subscription end is not re-determined for the set times.
4. The method according to claim 3, wherein the determining, based on the subscription configuration information, each target subscriber requesting the data stream to be distributed specifically includes:
and if the data stream to be distributed carries the subscriber indication information, determining each target subscriber in the subscribers indicated by the subscriber indication information based on the subscription configuration information.
5. The method of claim 1 or 2, wherein before the obtaining of the subscribed broadcast stream of the data stream requested by the subscriber, the method further comprises:
acquiring a subscription broadcast stream from a first message middleware, and acquiring a data stream to be distributed from a second message middleware, wherein the acquired subscription broadcast stream or one of the data streams to be distributed is labeled with a label to form a data stream to be processed; and
the acquiring of the subscription broadcast stream of the data stream requested by the subscriber specifically includes: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the label;
the acquiring of the data stream to be distributed sent by the data generating end specifically includes: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the label.
6. A data stream processing apparatus, comprising:
the broadcast stream processing unit is used for acquiring a subscription broadcast stream of a data stream requested by a subscription end and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
and the data stream processing unit is used for acquiring the data stream to be distributed sent by the data generating end, determining each target subscriber requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscriber.
7. The apparatus of claim 6, wherein the subscription broadcast stream further carries subscription type information of a subscriber, and the broadcast stream processing unit is further configured to: updating the subscription type information of each subscription terminal based on the subscription type information of the subscription terminal carried in the subscription broadcast stream; and
the data stream processing unit is specifically configured to:
determining the subscription type of each target subscriber based on the recorded subscription type information;
and respectively sending the data stream to be distributed to each target subscriber according to the data transmission protocol matched with the subscription type of each target subscriber.
8. The apparatus of claim 6 or 7, wherein the data stream processing unit is further to:
if the target subscriber terminal is not determined based on the subscription configuration information, discarding the data stream to be distributed; or storing the data stream to be distributed, re-determining whether a target subscription end exists based on the latest subscription configuration information when the storage duration of the data stream to be distributed reaches a duration threshold, and discarding the stored data stream to be distributed if the target subscription end is not re-determined for the set times.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1-6 are implemented when the program is executed by the processor.
10. A computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299080.7A CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299080.7A CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114520832A true CN114520832A (en) | 2022-05-20 |
| CN114520832B CN114520832B (en) | 2023-08-25 |
Family
ID=81594915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011299080.7A Active CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114520832B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516746A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for pushing service change information, device and system thereof |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | Posting/subscribing system for adding message queue models and working method thereof |
| CN104133871A (en) * | 2014-07-21 | 2014-11-05 | 广州华多网络科技有限公司 | Method and device for processing subscription information |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106603394A (en) * | 2013-12-05 | 2017-04-26 | 华为技术有限公司 | Method and device for realizing subscription notification |
| CN106612301A (en) * | 2015-10-22 | 2017-05-03 | 招商局国际信息技术有限公司 | Updated data pushing method and updated data pushing device |
| CN108366096A (en) * | 2018-01-17 | 2018-08-03 | 深圳市金立通信设备有限公司 | A kind of information subscribing method, terminal and computer readable storage medium |
| US20180241616A1 (en) * | 2017-02-17 | 2018-08-23 | Home Box Office, Inc. | Endpoint abstraction for service-to-service communication |
| CN109656725A (en) * | 2018-11-09 | 2019-04-19 | 北京字节跳动网络技术有限公司 | Message consumer's switching method, device, storage medium and electronic equipment |
| WO2019158049A1 (en) * | 2018-02-13 | 2019-08-22 | 中兴通讯股份有限公司 | Subscription information update method and apparatus |
| CN111163172A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Message processing system, method, electronic device and storage medium |
| US10833881B1 (en) * | 2017-11-06 | 2020-11-10 | Amazon Technologies, Inc. | Distributing publication messages to devices |
-
2020
- 2020-11-19 CN CN202011299080.7A patent/CN114520832B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516746A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for pushing service change information, device and system thereof |
| CN106603394A (en) * | 2013-12-05 | 2017-04-26 | 华为技术有限公司 | Method and device for realizing subscription notification |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | Posting/subscribing system for adding message queue models and working method thereof |
| CN104133871A (en) * | 2014-07-21 | 2014-11-05 | 广州华多网络科技有限公司 | Method and device for processing subscription information |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106612301A (en) * | 2015-10-22 | 2017-05-03 | 招商局国际信息技术有限公司 | Updated data pushing method and updated data pushing device |
| US20180241616A1 (en) * | 2017-02-17 | 2018-08-23 | Home Box Office, Inc. | Endpoint abstraction for service-to-service communication |
| US10833881B1 (en) * | 2017-11-06 | 2020-11-10 | Amazon Technologies, Inc. | Distributing publication messages to devices |
| CN108366096A (en) * | 2018-01-17 | 2018-08-03 | 深圳市金立通信设备有限公司 | A kind of information subscribing method, terminal and computer readable storage medium |
| WO2019158049A1 (en) * | 2018-02-13 | 2019-08-22 | 中兴通讯股份有限公司 | Subscription information update method and apparatus |
| CN109656725A (en) * | 2018-11-09 | 2019-04-19 | 北京字节跳动网络技术有限公司 | Message consumer's switching method, device, storage medium and electronic equipment |
| CN111163172A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Message processing system, method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114520832B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107566786B (en) | Method and device for acquiring monitoring video and terminal equipment | |
| US10439916B2 (en) | Client-side fault tolerance in a publish-subscribe system | |
| JP4521003B2 (en) | Apparatus and associated method for facilitating delivery and processing of push content | |
| CN109347968B (en) | Method, equipment and system for downloading data blocks of resource file | |
| US20110119388A1 (en) | Optimization of Multimedia Service Over an IMS Network | |
| CN111355971B (en) | Live streaming transmission method and device, CDN server and computer readable medium | |
| US11190576B2 (en) | File distribution and download method, distribution server, client terminal and system | |
| US20180189315A1 (en) | Method and device for calling a distributed file system | |
| US20150088995A1 (en) | Method and apparatus for sharing contents using information of group change in content oriented network environment | |
| CN101326493A (en) | Method and device for distributing load of multiprocessor server | |
| US11102319B2 (en) | Method, system and server for stream-pushing | |
| US8375124B1 (en) | Resumable upload for hosted storage systems | |
| CN110445723B (en) | Network data scheduling method and edge node | |
| CN107332894B (en) | Live broadcast method, device and system, server and storage medium | |
| CN108111630B (en) | Zookeeper cluster system and connection method and system thereof | |
| CN109547508B (en) | Method, device and system for realizing resource access | |
| US20180081746A1 (en) | Application message processing system, method, and application device | |
| CN112769671B (en) | Message processing method, device and system | |
| CN113010474A (en) | File management method, instant messaging method and storage server | |
| CN111835801B (en) | File downloading method, device, server, edge equipment, terminal and medium | |
| CN114520832B (en) | Data stream processing method, device, equipment and computer storage medium | |
| CN109618187B (en) | Video data acquisition method and device | |
| US8615548B1 (en) | System and method for deferred data downloading | |
| CN112995697B (en) | Streaming data recovery method, server, storage medium and computer equipment | |
| US20140149499A1 (en) | Remote request fulfillment and delivery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |