CN112751893A - Message track data processing method and device and electronic equipment - Google Patents
Message track data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN112751893A CN112751893A CN201911045244.0A CN201911045244A CN112751893A CN 112751893 A CN112751893 A CN 112751893A CN 201911045244 A CN201911045244 A CN 201911045244A CN 112751893 A CN112751893 A CN 112751893A
- Authority
- CN
- China
- Prior art keywords
- message
- track data
- service node
- data
- agent service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/562—Brokering proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The application discloses a method and a device for processing message track data and electronic equipment, wherein the method for processing the message track data comprises the following steps: acquiring message track data through an interceptor in a client; determining a message agent service node corresponding to the message track data; and sending the message track data to the message agent service node so as to store the message track data to a target storage area through the message agent service node. According to the method and the system, the interceptor in the client side is used for acquiring the message track data, so that the message agent service node does not need to additionally lose the performance of the server due to the need of retrieving, analyzing and calculating the related data of the message track.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for processing message track data, and an electronic device.
Background
Message Queue (MQ) is a container used for storing messages during transmission, and mainly plays the role of a "transit station" during a communication process of sending and receiving, and internally provides communication routing and ensures reliable delivery of the messages. After a message is sent from a producer, whether the message is sent successfully or consumed normally or not is a point which is required to be considered and concerned by a user when the user uses the message queue middleware. Without other more convenient and quick means, it is substantially difficult for a user to search a large-scale message queue cluster for the current processing state of a message. Therefore, a user using a message queue needs to have a mechanism to record and query message trace data of the current state of the message (whether sent successfully by the producer, whether consumed normally by the consumer, specific location saved) in the cluster environment.
Disclosure of Invention
In order to solve the above technical problem, embodiments of the present application provide a method and an apparatus for processing message track data, an electronic device, and a storage medium.
The message track data processing method provided by the embodiment of the application comprises the following steps:
acquiring message track data through an interceptor in a client;
determining a message agent service node corresponding to the message track data;
and sending the message track data to the message agent service node so as to store the message track data to a target storage area through the message agent service node.
In an optional embodiment of the present application, the sending the message track data to the message broker service node includes:
delivering the message trace data to a congestion queue;
acquiring the message track data from the blocking queue, generating an asynchronous request task based on the message track data, and submitting the asynchronous request task to a thread pool;
and processing the asynchronous request task by adopting an asynchronous thread mode through the thread pool so as to send the message track data to the message agent service node.
In an optional embodiment of the present application, the sending the message track data to the message broker service node includes:
and encapsulating the message track data into a target message format, and sending the message track data with the target message format to the message agent service node.
In an optional embodiment of the present application, the obtaining, by an interceptor in a client, message track data includes:
acquiring message track data through an interceptor in a client at a producer side; and/or the presence of a gas in the gas,
message trace data is obtained by an interceptor in a client on the consumer side.
In an optional embodiment of the present application, the determining a message broker service node corresponding to the message trajectory data includes:
and determining a first message agent service node corresponding to the message track data from a plurality of message agent service nodes through a load balancing algorithm, wherein the plurality of message agent service nodes all support storage of the message track data and common messages.
In an optional embodiment of the present application, each of the plurality of message broker service nodes creates the target storage area, and a theme attribute of the target storage area is a track theme.
In an optional embodiment of the present application, the determining a message broker service node corresponding to the message trajectory data includes:
and determining that the message agent service node corresponding to the message track data is a second message agent service node, wherein the second message agent service node only supports storage of the message track data.
In an optional embodiment of the present application, the second message broker service node creates the target storage area, and a theme attribute of the target storage area is a track theme.
An embodiment of the present application further provides a device for processing message track data, where the device includes:
the acquisition unit is used for acquiring message track data through an interceptor in the client;
the determining unit is used for determining a message agent service node corresponding to the message track data;
and the sending unit is used for sending the message track data to the message agent service node so as to store the message track data in a target storage area through the message agent service node.
In an optional embodiment of the present application, the sending unit includes:
a delivery subunit, configured to deliver the message track data to a congestion queue;
the obtaining subunit is configured to obtain the message track data from the blocking queue, generate an asynchronous request task based on the message track data, and submit the asynchronous request task to a thread pool;
and the sending subunit is configured to process the asynchronous request task in an asynchronous thread manner through the thread pool, so as to send the message track data to the message broker service node.
An embodiment of the present application provides an electronic device, including: the processor is used for calling and running the computer program stored in the memory, and the message track data processing method is executed.
An embodiment of the present application provides a computer-readable storage medium, which is configured to store a computer program, where the computer program enables a computer to execute the above-mentioned message track data processing method.
In the technical scheme of the embodiment of the application, message track data are obtained through an interceptor in a client; determining a message agent service node corresponding to the message track data; and sending the message track data to the message agent service node so as to store the message track data to a target storage area through the message agent service node. Therefore, the burden of analyzing and calculating the message track data by the message agent service node can be effectively reduced, and the performance of the server is not additionally lost by the message agent server due to the need of retrieving, analyzing and calculating the message track data.
Drawings
Fig. 1 is a schematic flowchart of a processing method of message track data according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a process of delivering/subscribing to a message from a message queue cluster by a producer/consumer process according to an embodiment of the present application;
FIG. 3 is a first diagram illustrating delivery of message track data to a message broker service node according to an embodiment of the present disclosure;
FIG. 4 is a second schematic diagram illustrating delivery of message track data to a message broker service node according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating that message track data is placed in a blocking queue and is asynchronously sent according to an embodiment of the present application;
fig. 6 is a schematic structural component diagram of a message track data processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a chip according to an embodiment of the present application.
Detailed Description
So that the manner in which the features and elements of the present embodiments can be understood in detail, a more particular description of the embodiments, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.
At present, a message queue has gradually become one of core means of internal communication of an enterprise IT system, and IT can be said that most of current large distributed internet service systems are constructed based on the message queue. It has a series of functions of low coupling, reliable delivery, broadcasting, flow control and final consistency, etc., and becomes one of the main means of asynchronous communication. The message queue is widely used as an important piece of middleware for constructing various currently important business systems (such as an order system, a payment system, a transaction system and other various management systems), so that the reliability and maintainability of the message queue are important considerations in system design, model selection and architecture construction.
Considering the factors of throughput, low delay and high availability of the whole message queue, almost all users use the message queue middleware (such as RockettMQ and Kafka) to deploy in a cluster mode when using the message queue middleware on a production environment. The scale of the message queue cluster is also related to the specific situation of the service, and the scale of the message queue cluster with large service volume can reach hundreds of nodes. However, for business systems (particularly financial or transactional systems) that use message queues, the reliability of the messages is of particular importance.
The message track data refers to complete link information formed by converging data such as time and place of each relevant node in the whole process of consumption processing of a message from a producer to a consumer. The complete track link of one message comprises a message producer, a message consumer and a message agent service party, related information is added in the track link in the process of processing the message in each part, the information is collected to obtain message track data, the current processing state of any message can be obtained through the message track data, and once the message cannot be delivered or subscribed normally, powerful data support can be provided for problem troubleshooting in a production environment through the obtained message track data.
In order to track and acquire the message track data, the message track data can be acquired or recorded in various ways, such as a plug-in of the message queue, a local log file in the message queue, an agent server in the message queue or third-party software, and the like. Four ways of acquiring or recording message track data are given below.
The first method is as follows:
the open source version RabbitMQ has the function of recording and inquiring message tracks, and message track data delivered/subscribed by each message in the RabbitMQ cluster can be tracked through a 'trading Log' plug-in of the RabbitMQ. Specifically, the message track data is stored and displayed in a common message format by sending some extra information (such as an IP address of a producer/consumer process, whether a message is delivered successfully, whether a message is subscribed successfully by a consumer, and other related information) of a delivery or subscription message of a producer or a consumer to a RabbitMQ cluster deployed at a server side in an asynchronous manner. When the method is adopted, the message track data and the common message are not isolated, and the message track data and the delivery and subscription of the common message are in a one-to-one correspondence relationship, so that a server deployed with the message agent service needs to store data files with the original 2 to 3 times volume capacity, and the common message and the message track data are stored on the same physical disk medium, so that the delivery/subscription performance, the throughput and the coefficient of the original common message are greatly influenced. In an actual production environment, when the service throughput is large, after a message track tracing plug-in "tracing Log" in a RabbitMQ is started to realize a message track tracing function, the performance of the whole RabbitMQ cluster is sharply reduced, the delay of message delivery/subscription of the RabbitMQ message queue cluster is greatly increased, and the normal use of a message queue by a service system is influenced, so that the implementation method cannot be applied to the actual production environment.
The second method comprises the following steps:
the complete consumption track of the service message is retrieved and obtained through the message sending and receiving conditions contained in the local log files of the message consumer, the message producer and the message agent and the service content of the associated service message, and the abnormal message fault location is realized according to the retrieval of the service key words input by the user. When the method is adopted, the retrieval of the message track data needs to be completed from the local log files of the producer, the consumer and the message agent according to the keywords, and the three log files are all large in the actual production environment, so that a large space is needed for storing the three log files, a considerable part of the log files are data irrelevant to the service message, a large amount of time is consumed during the search, and a great amount of extra operation and maintenance cost is brought to the fault of the positioning message.
The third method comprises the following steps:
at the agent server side of the message queue, by determining and acquiring 'production information of a message production stage', 'storage information of a message storage stage' and 'consumption information about a message consumption stage', and simultaneously storing the information into a corresponding message channel, the troubleshooting positioning can be conveniently carried out in the troubleshooting process according to the recorded information. When the implementation mode is adopted, the message track data collection and arrangement are mainly completed at the message agent service end, the related data are integrated and stored in the corresponding message channel, the scheme can increase the calculation and memory occupation cost of the message agent service node in the message queue, and the throughput of the message agent service node and the overall efficiency of message processing are reduced to a certain extent.
The method is as follows:
the information which is generated by the service system and can be used for representing the message track is stored in a database according to an agreed format after being analyzed, and meanwhile, the query result of the message track data is displayed according to key fields (such as the number of the service system, the unique identification of the message in the service system, the service type, the source user number, the destination user number, the message submitting time and the message issuing time) by combining a configured logic relation table among all service systems. When the method is adopted, the data related to the business message track is retrieved in the business system by means of third-party software such as a database, and the storage of the message track data is completed. The implementation method depends heavily on the service system, and when the service system runs abnormally, the corresponding service message track data is lost. Meanwhile, when the message track data is stored by relying on software in a third party such as a database, if the operation of the database environment is unstable, such as a database service failure, or a database deadlock occurs, the message track data cannot be queried in time.
Based on the above 4 modes of description and the analysis of the problems, various embodiments of the present application are proposed.
According to the method and the device, the message track data in the message queue are obtained through the interceptor in the client, the message track data can be obtained according to the self capacity of the message queue, third-party software is not needed, other additional information is not included in the obtained message track data, the message track data can be quickly and efficiently retrieved when the fault of the message queue is positioned, meanwhile, the message track data can be stored in different modes according to the message throughput condition in the message queue cluster, and the delivery and subscription of common messages in the message queue cluster cannot be influenced.
Fig. 1 is a schematic flow chart of a processing method for providing message track data according to an embodiment of the present application, and as shown in fig. 1, the processing method for message track data includes the following steps:
step 101: and acquiring message track data through an interceptor in the client.
In an optional embodiment of the present application, the obtaining, by an interceptor in a client, message track data includes:
acquiring message track data through an interceptor in a client at a producer side; and/or the presence of a gas in the gas,
message trace data is obtained by an interceptor in a client on the consumer side.
Fig. 2 is a schematic diagram of a process of delivering/subscribing to messages from a cluster of message queues by a producer/consumer process.
In one embodiment, the acquisition of the message trace data may be implemented by an interceptor of a client on the producer and/or consumer side. Here, entry points of the delivery and subscription message interceptors may be defined at the clients of the message queues integrated by the producer and consumer processes themselves in fig. 2, respectively, for implementing aggregation and collection of message trajectory data before and after delivery and subscription messages. For example, a Hook method of delivering/subscribing to messages can be used as an entry point of the interceptor, so that the purpose of efficiently acquiring message track data at the client is achieved.
Specifically, the information included in the message track data acquired by the interceptor in the client is shown in table 1 below:
TABLE 1
It should be noted that, in the embodiment of the present application, the acquisition of the message track data is set at the client, and the following points are mainly considered:
1. the producer and the consumer process can integrate the client of the message queue to realize the delivery and subscription of the message, and the interceptor of the client can obtain the message track data, so that the burden of the message agent service node can be reduced, the scheme of the application can be more reasonable and light, and the message agent service node does not need to additionally lose the performance of the server because of the need of retrieving, analyzing and calculating the message track data.
2. After the client collects the message track data, the client can package the message track data into a target message format, for example, into a format of a common message in the message queue, and deliver the message track data to a target storage area of the message agent service node by means of a delivery mechanism of the message queue. Specifically, different target storage areas have different theme attributes, and a user can determine on which message proxy service nodes message track data is stored by configuring a track theme for storing a message track at a client according to the requirement of service throughput.
In the embodiment of the application, the message track data is acquired through the interceptor of the client, service data irrelevant to the message track data does not need to be additionally stored, the capacity of the stored data can be reduced, the retrieval efficiency can be improved when the message track data is retrieved through keywords, and the fault in the message queue cluster can be quickly positioned. In addition, the technical scheme of the embodiment of the application can be realized according to the self capacity of the message queue middleware, a third-party data retrieval tool is not needed, the complexity of operation and maintenance of the message queue cluster can be reduced, and meanwhile, the instability caused by the introduction of the third-party middleware is reduced.
Step 102: and determining a message agent service node corresponding to the message track data.
In an optional embodiment of the present application, the determining a message broker service node corresponding to the message trajectory data includes:
and determining a first message agent service node corresponding to the message track data from a plurality of message agent service nodes through a load balancing algorithm, wherein the plurality of message agent service nodes all support storage of the message track data and common messages.
Correspondingly, each message agent service node in the plurality of message agent service nodes is created with the target storage area, and the theme attribute of the target storage area is a track theme.
Specifically, fig. 3 is a first schematic diagram illustrating the storage of the message track data in the target storage area. In fig. 3, when the message broker service process is started, a trace topic (TraceTopic) dedicated to storing message trace data is created by default, correspondingly, a target storage area is created in each of a plurality of message broker service nodes (message broker service node 1, message broker service node 2, message broker service node 3, and message broker service node 4), where a topic attribute of the target storage area is TraceTopic, and it should be noted that each message broker service node (message broker service node 1, message broker service node 2, message broker service node 3, and message broker service node 4) supports storage of message trace data and general messages. As shown in fig. 3, After the interceptor of the producer client acquires the message trace data and encapsulates the message trace data into the target message format, the producer process 1 includes a request for delivering the message trace data, and delivers the message trace data to the TraceTopic of the first message broker service node as a common message in the Before ()/After () method of the client interceptor, thereby completing the asynchronous transmission of the message trace data. Similarly, After the interceptor of the consumer client acquires the message track data and encapsulates the message track data into a target message format, the consumer process 1 includes a request for subscribing the message track data, and delivers the message track data to the TraceTopic of the first message broker service node as a common message in the Before ()/After () method of the client interceptor, thereby completing the asynchronous transmission of the message track data. Wherein the address of the first message broker service node is determined by the client from the plurality of message broker service nodes through a load balancing algorithm.
In this embodiment, when storing the message track data, the common message and the message track data are not physically distinguished and isolated at the message broker service node, so that there is an influence on the performance overhead of the server disk IO when reading and writing data files such as the common message and the message track data on the disk. When the throughput of the service message in the message queue cluster continuously becomes high, the writing and reading of the message track data can increase the IO overhead of the physical disk of the server, thereby causing the delay of the delivery and subscription of the common message. Therefore, the embodiment is suitable for the scene that the message queue cluster has low service message throughput.
In another optional implementation manner of this application, the determining a message broker service node corresponding to the message trajectory data includes:
and determining that the message agent service node corresponding to the message track data is a second message agent service node, wherein the second message agent service node only supports storage of the message track data.
Correspondingly, the second message agent service node creates the target storage area, and the theme attribute of the target storage area is a track theme.
Specifically, fig. 4 is a schematic diagram of storing the message track data in the target storage area. In fig. 4, isolation from the normal message is achieved by selecting one of the message proxy service nodes (message proxy service node 1, message proxy service node 2, message proxy service node 3, message proxy service node 4, and message proxy service node 5) in the message queue cluster, which is the second message proxy service node, to be dedicated to storing message trace data. Optionally, a second message broker service node (message broker service node 5) dedicated to storing message trajectory data may be set when the message broker service node is started by configuring information parameters, and accordingly, a target area is created on the second message broker service node (message broker service node 5), where a subject attribute of the target area is TraceTopic. In this embodiment, only on the second message broker service node (message broker service node 5), tracetopoc that stores message trace data is created by default. Accordingly, the message broker service node calculated when the client performs load balancing algorithm selection according to TraceTopic will always be the address of the second message broker service node (message broker service node 5). Therefore, in fig. 4, After the interceptor of the producer client acquires the message trace data and encapsulates the message trace data into the target message format, the producer process 1 includes a request for delivering the message trace data, and delivers the message trace data to the TraceTopic of the second message broker service node (message broker service node 5) as a common message in the Before ()/After () method of the client interceptor, thereby completing the asynchronous transmission of the message trace data. Similarly, After the interceptor of the consumer client acquires the message track data and encapsulates the message track data into the target message format, the consumer process 1 includes a request for subscribing the message track data, and delivers the message track data to the TraceTopic of the second message broker service node (message broker service node 5) as a common message in the Before ()/After () method of the client interceptor, thereby completing the asynchronous transmission of the message track data. It should be noted that the second message broker service node (message broker service node 5) is only used for storing the message track data, and does not store the common messages in the message queue, and by adopting the embodiment, the message track data can be isolated from the common messages in the message queue.
The embodiment is an isolation mode for storing message track data, in the embodiment, the message track data is separately stored on a second message agent service node in a space time-switching mode, so that the problems of low throughput and performance caused by the fact that read-write data contend for disk IO (input/output) due to the fact that common messages and the message track data in a message queue are stored in the same message agent service node in a mixed mode when a common mode is adopted can be effectively solved, and the method is suitable for a service scene with high message throughput in a message queue cluster.
Step 103: and sending the message track data to the message agent service node so as to store the message track data to a target storage area through the message agent service node.
In an optional implementation manner of this application, the sending the message track data to the message broker service node includes:
delivering the message trace data to a congestion queue;
acquiring the message track data from the blocking queue, generating an asynchronous request task based on the message track data, and submitting the asynchronous request task to a thread pool;
and processing the asynchronous request task by adopting an asynchronous thread mode through the thread pool so as to send the message track data to the message agent service node.
Specifically, fig. 5 is a schematic diagram of placing message trace data into a blocking queue and performing asynchronous transmission. In the embodiment of the application, the interceptor at the client of the producer and/or the consumer packages the message track data obtained by aggregating the complete link information of the delivery/subscription messages acquired by the interceptor into a data packet according to the target message format, and then puts the data packet into the blocking queue. And meanwhile, starting an asynchronous thread at the client to acquire a data packet from the blocking queue and sending the data packet to a target storage area of the message agent service node in an asynchronous mode.
In fig. 5, a "producer/consumer model" and a "blocking queue" are adopted, a hook function (i.e., clientsendmessagehookimpr) of a producer client is mainly to take a client delivering a message as a producer thread to add an acquired message track data packet into the blocking queue (i.e., traceContextQueue), and similarly, a hook function (i.e., clientconsumermessagehookimpr) of a consumer client is mainly to take a client subscribing to a message as a consumer thread to add an acquired message track data packet into the blocking queue, and when the client starts an asynchronous thread (i.e., MQ-asystem dispatcher), the consumer client acquires message track data from the blocking queue and encapsulates the message track data into an asynchronous request task (i.e., asyncaprederrerrequest) to submit the asynchronous request task to a thread pool. The main execution flow of the asynchronous Request task is to finally call a default producer instance (traceProducer) inside the client to deliver the encapsulated message track data to a subject dedicated for storing the message track, and specifically, a Request thread in a thread pool asynchronously executes to send the message track data to a target storage area in a remote message agent service node.
By adopting the method and the system, the message track data acquired by the client can be asynchronously sent to the message agent service node in a specific message format.
In this embodiment of the application, the sending the message track data to the message broker service node includes:
and encapsulating the message track data into a target message format, and sending the message track data with the target message format to the message agent service node. The target message format may be a common message format in a message queue or a message format set by a user as required.
According to the technical scheme, when the client delivers or subscribes the message, the message track data is acquired through the interceptor of the client, so that the burden of analyzing and calculating the message track data by the message agent service node can be effectively reduced, and the whole scheme is more reasonable and lighter. The message broker service node does not need to additionally lose the performance of the server in retrieving, parsing and computing the message trace data. In practical application, the client may encapsulate the message track data into a target message format, such as a common message format, by using the delivery capability of the message queue itself, and deliver the message track data to the message proxy service node, so as to store the message track data in the target storage area through the message proxy service node. The user can determine which message agent service nodes the message track data is stored on by configuring a self-defined track theme for storing the message track data at the client according to the requirement of the traffic.
In addition, in the technical solution of the embodiment of the present application, two different modes ("normal mode" and "isolated mode") are adopted to adapt to the service scenario requirements of different message throughputs in the message queue cluster. The common mode mainly mixes and stores common messages and message track data on the same message agent service node, and can be suitable for scenes with low business message throughput; in the isolation mode, IO physical isolation between the common message and the message track data is realized by selecting a message agent service node which is specially used for storing the message track data in the whole message queue cluster, so that even in a high-throughput service scene, no performance influence is generated on delivery and subscription of the common message after the message track tracking function is started.
Fig. 6 is a schematic structural composition diagram of a message track data processing apparatus according to an embodiment of the present application, and as shown in fig. 6, the apparatus includes:
an obtaining unit 601, configured to obtain message trajectory data through an interceptor in a client;
a determining unit 602, configured to determine a message proxy service node corresponding to the message trajectory data;
a sending unit 603, configured to send the message track data to the message broker service node, so as to store the message track data in a target storage area through the message broker service node.
In an optional embodiment of the present application, the sending unit is further configured to deliver the message track data to a congestion queue; acquiring the message track data from the blocking queue, generating an asynchronous request task based on the message track data, and submitting the asynchronous request task to a thread pool; and processing the asynchronous request task by adopting an asynchronous thread mode through the thread pool so as to send the message track data to the message agent service node.
In an optional implementation manner of this application, the sending unit is further configured to encapsulate the message track data into a target message format, and send the message track data in the target message format to the message broker service node.
In an optional embodiment of the present application, the obtaining unit is specifically configured to obtain message trajectory data through an interceptor in a client on a producer side; and/or, obtaining the message track data through an interceptor in a client of the consumer side.
In an optional embodiment of the present application, the determining unit is specifically configured to determine, by using a load balancing algorithm, a first message proxy service node corresponding to the message trajectory data from a plurality of message proxy service nodes, where the plurality of message proxy service nodes all support storage of the message trajectory data and a common message.
Optionally, each message broker service node in the plurality of message broker service nodes is created with the target storage area, and a theme attribute of the target storage area is a track theme.
In an optional implementation manner of this application, the determining unit is further configured to determine that the message broker service node corresponding to the message trajectory data is a second message broker service node, where the second message broker service node only supports storage of the message trajectory data.
Optionally, the second message broker service node creates the target storage area, and a theme attribute of the target storage area is a track theme.
Those skilled in the art will understand that the implementation functions of each unit in the processing device of the message track data shown in fig. 6 can be understood by referring to the related description of the aforementioned data processing method. The functions of the units in the data processing apparatus shown in fig. 6 may be implemented by a program running on a processor, or may be implemented by specific logic circuits.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may be a server, the electronic device includes the message trace data processing apparatus shown in fig. 6, the electronic device 700 shown in fig. 7 includes a processor 710, and the processor 710 may call and execute a computer program from a memory to implement the method in the embodiment of the present application.
Optionally, as shown in fig. 7, the electronic device 700 may also include a memory 720. From the memory 720, the processor 710 can call and run a computer program to implement the method in the embodiment of the present application.
The memory 720 may be a separate device from the processor 710, or may be integrated into the processor 710.
Optionally, as shown in fig. 7, the electronic device 700 may further include a transceiver 730, and the processor 710 may control the transceiver 730 to communicate with other devices, and specifically, may transmit information or data to the other devices or receive information or data transmitted by the other devices.
The transceiver 730 may include a transmitter and a receiver, among others. The transceiver 730 may further include an antenna, and the number of antennas may be one or more.
Optionally, the electronic device 700 may specifically be a network device in the embodiment of the present application, and the electronic device 700 may implement a corresponding process implemented by the network device in each method in the embodiment of the present application, which is not described herein again for brevity.
Optionally, the electronic device 700 may specifically be a mobile terminal/terminal device according to this embodiment, and the electronic device 700 may implement a corresponding process implemented by the mobile terminal/terminal device in each method according to this embodiment, which is not described herein again for brevity.
Fig. 8 is a schematic structural diagram of a chip of an embodiment of the present application. The chip 800 shown in fig. 8 includes a processor 810, and the processor 810 can call and run a computer program from a memory to implement the method in the embodiment of the present application.
Optionally, as shown in fig. 8, chip 800 may further include a memory 820. From the memory 820, the processor 810 can call and run a computer program to implement the method in the embodiment of the present application.
The memory 820 may be a separate device from the processor 810 or may be integrated into the processor 810.
Optionally, the chip 800 may further include an input interface 830. The processor 810 may control the input interface 830 to communicate with other devices or chips, and specifically, may obtain information or data transmitted by other devices or chips.
Optionally, the chip 800 may further include an output interface 840. The processor 810 can control the output interface 840 to communicate with other devices or chips, and in particular, can output information or data to other devices or chips.
Optionally, the chip may be applied to the network device in the embodiment of the present application, and the chip may implement the corresponding process implemented by the network device in each method in the embodiment of the present application, and for brevity, details are not described here again.
Optionally, the chip may be applied to the mobile terminal/terminal device in the embodiment of the present application, and the chip may implement the corresponding process implemented by the mobile terminal/terminal device in each method in the embodiment of the present application, and for brevity, no further description is given here.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip or a system-on-chip, etc.
It should be understood that the processor of the embodiments of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
It will be appreciated that the memory in the embodiments of the subject application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), Enhanced Synchronous SDRAM (ESDRAM), Synchronous link SDRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
It should be understood that the above memories are exemplary but not limiting illustrations, for example, the memories in the embodiments of the present application may also be Static Random Access Memory (SRAM), dynamic random access memory (dynamic RAM, DRAM), Synchronous Dynamic Random Access Memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (enhanced SDRAM, ESDRAM), Synchronous Link DRAM (SLDRAM), Direct Rambus RAM (DR RAM), and the like. That is, the memory in the embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.
The embodiment of the application also provides a computer readable storage medium for storing the computer program.
Optionally, the computer-readable storage medium may be applied to the network device in the embodiment of the present application, and the computer program enables the computer to execute the corresponding process implemented by the network device in each method in the embodiment of the present application, which is not described herein again for brevity.
Optionally, the computer-readable storage medium may be applied to the mobile terminal/terminal device in the embodiment of the present application, and the computer program enables the computer to execute the corresponding process implemented by the mobile terminal/terminal device in each method in the embodiment of the present application, which is not described herein again for brevity.
Embodiments of the present application also provide a computer program product comprising computer program instructions.
Optionally, the computer program product may be applied to the network device in the embodiment of the present application, and the computer program instructions enable the computer to execute corresponding processes implemented by the network device in the methods in the embodiment of the present application, which are not described herein again for brevity.
Optionally, the computer program product may be applied to the mobile terminal/terminal device in the embodiment of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the mobile terminal/terminal device in the methods in the embodiment of the present application, which are not described herein again for brevity.
The embodiment of the application also provides a computer program.
Optionally, the computer program may be applied to the network device in the embodiment of the present application, and when the computer program runs on a computer, the computer is enabled to execute the corresponding process implemented by the network device in each method in the embodiment of the present application, and for brevity, details are not described here again.
Optionally, the computer program may be applied to the mobile terminal/terminal device in the embodiment of the present application, and when the computer program runs on a computer, the computer is enabled to execute the corresponding process implemented by the mobile terminal/terminal device in each method in the embodiment of the present application, which is not described herein again for brevity.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A method for processing message track data, the method comprising:
acquiring message track data through an interceptor in a client;
determining a message agent service node corresponding to the message track data;
and sending the message track data to the message agent service node so as to store the message track data to a target storage area through the message agent service node.
2. The method of claim 1, wherein sending the message trace data to the message broker service node comprises:
delivering the message trace data to a congestion queue;
acquiring the message track data from the blocking queue, generating an asynchronous request task based on the message track data, and submitting the asynchronous request task to a thread pool;
and processing the asynchronous request task by adopting an asynchronous thread mode through the thread pool so as to send the message track data to the message agent service node.
3. The method of claim 2, wherein sending the message trace data to the message broker service node comprises:
and encapsulating the message track data into a target message format, and sending the message track data with the target message format to the message agent service node.
4. The method of claim 1, wherein the obtaining of the message trace data by the interceptor in the client comprises:
acquiring message track data through an interceptor in a client at a producer side; and/or the presence of a gas in the gas,
message trace data is obtained by an interceptor in a client on the consumer side.
5. The method according to any one of claims 1 to 4, wherein the determining a message broker service node to which the message track data corresponds comprises:
and determining a first message agent service node corresponding to the message track data from a plurality of message agent service nodes through a load balancing algorithm, wherein the plurality of message agent service nodes all support storage of the message track data and common messages.
6. The method of claim 5, wherein each of the plurality of message broker service nodes creates the target storage area, and wherein a topic attribute of the target storage area is a track topic.
7. The method according to any one of claims 1 to 4, wherein the determining a message broker service node to which the message track data corresponds comprises:
and determining that the message agent service node corresponding to the message track data is a second message agent service node, wherein the second message agent service node only supports storage of the message track data.
8. The method of claim 7, wherein the second message broker service node creates the target storage area, and wherein a topic attribute of the target storage area is a track topic.
9. An apparatus for processing message track data, the apparatus comprising:
the acquisition unit is used for acquiring message track data through an interceptor in the client;
the determining unit is used for determining a message agent service node corresponding to the message track data;
and the sending unit is used for sending the message track data to the message agent service node so as to store the message track data in a target storage area through the message agent service node.
10. The apparatus of claim 9, wherein the sending unit comprises:
a delivery subunit, configured to deliver the message track data to a congestion queue;
the obtaining subunit is configured to obtain the message track data from the blocking queue, generate an asynchronous request task based on the message track data, and submit the asynchronous request task to a thread pool;
and the sending subunit is configured to process the asynchronous request task in an asynchronous thread manner through the thread pool, so as to send the message track data to the message broker service node.
11. An electronic device, comprising: a processor and a memory for storing a computer program, the processor being configured to invoke and execute the computer program stored in the memory to perform the method of any of claims 1 to 8.
12. A computer-readable storage medium for storing a computer program which causes a computer to perform the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911045244.0A CN112751893A (en) | 2019-10-30 | 2019-10-30 | Message track data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911045244.0A CN112751893A (en) | 2019-10-30 | 2019-10-30 | Message track data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112751893A true CN112751893A (en) | 2021-05-04 |
Family
ID=75640608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911045244.0A Withdrawn CN112751893A (en) | 2019-10-30 | 2019-10-30 | Message track data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112751893A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434361A (en) * | 2021-06-23 | 2021-09-24 | 平安国际智慧城市科技股份有限公司 | System abnormity warning method, system, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649766A (en) * | 2016-12-27 | 2017-05-10 | 北京锐安科技有限公司 | Message handling method based on kafka |
| CN107609920A (en) * | 2017-09-30 | 2018-01-19 | 广州云移信息科技有限公司 | A kind of orbit generation method and system |
| CN109688200A (en) * | 2018-11-30 | 2019-04-26 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, device and equipment |
| CN110362721A (en) * | 2018-04-08 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Processing method, system, device and the electronic equipment of message traces information |
-
2019
- 2019-10-30 CN CN201911045244.0A patent/CN112751893A/en not_active Withdrawn
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649766A (en) * | 2016-12-27 | 2017-05-10 | 北京锐安科技有限公司 | Message handling method based on kafka |
| CN107609920A (en) * | 2017-09-30 | 2018-01-19 | 广州云移信息科技有限公司 | A kind of orbit generation method and system |
| CN110362721A (en) * | 2018-04-08 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Processing method, system, device and the electronic equipment of message traces information |
| CN109688200A (en) * | 2018-11-30 | 2019-04-26 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, device and equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434361A (en) * | 2021-06-23 | 2021-09-24 | 平安国际智慧城市科技股份有限公司 | System abnormity warning method, system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111625452B (en) | Flow playback method and system | |
| US20170083579A1 (en) | Distributed data processing method and system | |
| CN109521956B (en) | Cloud storage method, device, equipment and storage medium based on block chain | |
| CN112148674A (en) | Log data processing method and device, computer equipment and storage medium | |
| CN110196843B (en) | File distribution method based on container cluster and container cluster | |
| CN109522462B (en) | Cloud query method, device, equipment and storage medium based on block chain | |
| CN108833500B (en) | Service calling method, service providing method, data transmission method and server | |
| CN112751893A (en) | Message track data processing method and device and electronic equipment | |
| WO2023231723A1 (en) | Streaming media data processing method and system | |
| CN116700779A (en) | Code dynamic adjustment method, device, equipment, medium and program product | |
| CN114610449B (en) | Multi-cluster resource operation method and system based on unified request entry | |
| CN112445861A (en) | Information processing method, device, system and storage medium | |
| US20230188619A1 (en) | Methods and systems for event management | |
| CN115426375A (en) | Data processing method and data processing system | |
| US11706097B2 (en) | Task processing method applied to network topology, electronic device and storage medium | |
| CN112615773B (en) | Message processing method and system | |
| CN108805741B (en) | Fusion method, device and system of power quality data | |
| CN112416980A (en) | Data service processing method, device and equipment | |
| CN103856359A (en) | Method and system for obtaining information | |
| CN111538660B (en) | Method and device for counting coverage rate of program codes in cloud container | |
| CN115529288B (en) | Message pushing method, device and computer equipment | |
| CN113051152B (en) | Task data generation method and device and processing equipment | |
| CN117675930A (en) | Request processing system, method, device, computer equipment and storage medium | |
| CN117424809A (en) | Information sending method and device based on multiple instances and storage medium | |
| CN116860881A (en) | Data synchronization method, data synchronization device and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210504 |
|
| WW01 | Invention patent application withdrawn after publication |