CN114979259A - Message queue agent device - Google Patents
Message queue agent device Download PDFInfo
- Publication number
- CN114979259A CN114979259A CN202210335428.6A CN202210335428A CN114979259A CN 114979259 A CN114979259 A CN 114979259A CN 202210335428 A CN202210335428 A CN 202210335428A CN 114979259 A CN114979259 A CN 114979259A
- Authority
- CN
- China
- Prior art keywords
- message
- agent
- interface
- message queue
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003795 chemical substances by application Substances 0.000 claims description 112
- 238000000034 method Methods 0.000 claims description 36
- 238000012545 processing Methods 0.000 claims description 25
- 239000003999 initiator Substances 0.000 claims description 20
- 238000004806 packaging method and process Methods 0.000 claims description 2
- 230000000977 initiatory effect Effects 0.000 claims 1
- 239000007858 starting material Substances 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 235000010627 Phaseolus vulgaris Nutrition 0.000 description 4
- 244000046052 Phaseolus vulgaris Species 0.000 description 4
- 238000012827 research and development Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 2
- 241000136406 Comones Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2212/00—Encapsulation of packets
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to a message queue agent device, which is characterized by comprising a sending end agent, a receiving end agent and an agent internal protocol, wherein the sending end agent comprises a sending end agent interface and a plurality of sending end adapters; for each message queue in the agent internal protocol, at least one sending end adapter and at least one receiving end processor in the sending end agent and the receiving end agent are mutually associated and interacted or communicated, through the application of the invention device, the calling codes of the message queues do not need to be written, the development cost can be reduced, the difference in use caused by different personnel levels can be eliminated, the messages can be better ensured not to be lost due to downtime, and the used message queue services can be switched at any time, and the like.
Description
Technical Field
The invention relates to the field of software, in particular to a message queue agent device.
Background
The message queue middleware is an important component in a distributed system and mainly solves the problems of application coupling, asynchronous messages, traffic cut and the like. There are two steps for the use of various message queues. The method comprises the steps of submitting a message queue to a message queue middleware, acquiring the message queue from the message queue middleware, and calling a service processing method to process the message. Currently, there are ActiveMQ, RabbitMQ, ZeroMQ, Kafka, MetaMQ, RocktMQ, Redis, etc. in the market where more messages are used. Both of which are provided with clients for message submission and message retrieval. But different message queue middleware, their clients are very different in usage, which results in: 1. developers need to adapt to different clients if they use their client code directly. This undoubtedly imposes a huge burden on developers. 2. Developers have different levels, and when the developers directly use the clients, various situations, even serious errors, are inevitable, so that messages are lost. 3. The client directly using the message queue is not beneficial to the management of the message queue, and the call chain log can be disconnected. In addition, as the research and development group of an internet company becomes larger, it is inevitable to introduce different message queues. There are many differences in the functions and uses of these message queues, which increases the maintenance cost of the product at the later stage and the learning cost of the developers.
Disclosure of Invention
The invention mainly aims to overcome part of defects in the prior art and provides a message queue agent device which is characterized by comprising a sending end agent, a receiving end agent and an agent internal protocol, wherein the sending end agent comprises a sending end agent interface and a plurality of sending end adapters; for each message queue in the agent internal protocol, at least one sending end adapter and at least one receiving end processor in the sending end agent and the receiving end agent are associated with each other to interact or communicate;
preferably, the sender-side adapter is capable of sending the message obtained from the sender-side agent interface to a corresponding topic of a message queue corresponding to an agent internal protocol; the receiving end processor can be taken down from a message topic master-slave message queue to be processed by the receiving end agent interface, and then the receiving end agent interface is called to process specific services;
preferably, after the sending-end agent obtains the message from the sending-end agent interface, the sending-end agent encapsulates the message and then sends the encapsulated message; the receiving end agent device firstly analyzes the packaged information and then calls an interface of the receiving end agent device to process specific services;
preferably, the intra-broker protocols include any two or more of ActiveMQ, RabbitMQ, ZeroMQ, Kafka, MetaMQ, rocktmq, or Redis protocols;
preferably, the device further comprises a sending end agent and a receiving end agent, wherein the sending end agent and the receiving end agent comprise a submitting message end agent factory and a message obtaining and processing starter, the submitting message end agent factory can automatically generate a submitting message agent of the service interface, and the message obtaining and processing starter can start a specific client end for obtaining messages and call a specific service implementation to process the service;
preferably, when a submission message agent factory can automatically generate a submission message agent of a service interface, the submission message agent factory includes acquiring a currently called context variable, acquiring a variable transmitted by a currently called method, encrypting the method variable, packaging all variables, calling a specific submission client, and extracting the packaged message into a message queue;
preferably, when the initiator for acquiring and processing the message works, after acquiring the message through a specific initiator, namely a message acquisition client, the initiator reads header information from the message and stores the header information in a currently called context, and then decrypts method parameters according to encryption parameters in the header information; then calling a service interface to specifically realize processing service; judging whether normal processing is finished or not, if not, re-acquiring the current message and processing again, otherwise, removing the current message from the message queue by the message acquisition client and acquiring the next message;
preferably, the interface of the sending-end agent and the interface of the sending-end agent are the same interface;
preferably, when the message queue agent device is applied, the method includes adding a label to the sending end agent interface or the receiving end agent interface, and configuring the submitting message end agent factory and a starter for obtaining and processing messages;
the advantageous effects of the present invention will be described in the detailed description.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts;
FIG. 1 is a schematic diagram of the overall structure of a message queue agent device;
FIG. 2 is a diagram of a generic agent architecture;
FIG. 3 is a schematic diagram of a submit message side agent factory structure and process;
fig. 4 is a schematic diagram of the starter structure and processing.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative effort belong to the protection scope of the present invention;
in addition, when only the structure and the composition scheme related to the solution of the technical problem are described in the present invention, the technical features that can be determined by the known structures and processes or the common knowledge in combination with the drawings are not described below, but are not equal to the technical scheme, and should not be a reason for insufficient disclosure;
referring to fig. 1: the overall structure of the message queue agent device is shown in a schematic diagram in a preferred embodiment. The message queue agent device comprises a sending end agent, a receiving end agent and an agent internal protocol. The sending end agent device consists of a sending end agent device interface and various sending end adapters, wherein the various sending end adapters correspond to the agent device internal protocols or message queues one by one respectively. The sending end proxy interface is provided for developers to use, and the sending end adapter sends the messages obtained from the sending end proxy interface to the corresponding subjects of the corresponding message queues. The sending end adapter is written according to different message queues in advance. A developer only needs to write a specific message queue name on a configuration file;
the receiving end proxy device is composed of a receiving end proxy device interface and a receiving end processor of various message queues. And the receiving end processor is in charge of taking down the message topic to be processed from the receiving end agent interface from the master-slave message queue and then calling the receiving end agent interface to process the specific service. The receiving end processor writes in advance according to different message queues. A developer only needs to write a specific message queue name on a configuration file;
preferably, the sender agent interface of the sender agent and the receiver agent interface of the receiver agent may be the same interface, except that the agent interface is implemented at the sender as a send adapter that invokes various message queues. The proxy interface is the specific service processing at the receiving end;
an agent internal protocol, a communication protocol of the sending-end agent and the receiving-end agent. After the sending-end agent obtains the message from the sending-end agent interface, the sending-end agent encapsulates the message and then sends the encapsulated message. The receiving end proxy device needs to analyze the packaged message and then call the receiving end proxy device interface to process the specific service. The format of the protocol is: { message header: { requested, message creation time, method name signature, message encryption method }, message body: "message ciphertext". Wherein: 1, requesting is a calling chain id, a sending end obtains from a log context (MDC), and a receiving end newly stores the id in the log context;
therefore, through the message queue agent device, developers do not need to directly use different client codes and adapt to different clients, the problem that the messages are lost due to the fact that the developers have different levels and are difficult to avoid various conditions and even serious errors when the developers directly use the clients is solved, research and development personnel can switch/use different message queues only by modifying the configuration of the agent, the maintenance cost of codes and the learning cost of the research and development personnel are reduced, and undifferentiated message queue agent is achieved;
referring to fig. 2, a generic agent structure is employed by the sender agent and the receiver agent in the preferred embodiment. The system comprises a service interface A, a submitting message end agent factory E and a message acquiring and processing starter D. Specifically, the sending-end agent is a sending-end agent interface, and the receiving-end agent is a receiving-end agent interface. And the submission message end agent factory automatically generates a submission message agent of the service interface after the project is started. And after the project is started, the starting starter starts a specific client for acquiring the message to acquire the message and calls a specific service implementation B to process the service. The developer only needs to design the service interface a and the specific service implementation B. Taking Worker as an example, the originally injected variable of the variable service is an object of the hello service impl type. Now whenever the configuration of an item is modified, the injected variable becomes an object of the message queue commit agent type. Any method called in service in the Worker at this time becomes a commit message to the queue. Then, the configuration of the initiator is added for hello serviceimpl. The initiator will automatically initiate the client that obtains the message, take the message from the queue and call the corresponding method of HelloServiceImpl to process. For the initiator agent, the message-side agent factory E and the initiator D of message acquisition and processing may be integrated/contained within various initiator adapters. For the receiving end agent, a message end agent factory E and a message acquiring and processing starter D can be integrated/contained in the receiving end processors of various message queues;
referring to fig. 3, a schematic diagram of a commit messenger agent factory structure and process is shown, the commit messenger agent factory including read interface information and a particular sub-factory generating a commit agent. Each message queue of a specific sub-factory is written in advance, and which message queue needs to be used in a project is directly added into corresponding configuration. When a project is started, the factory generates a corresponding submitting end agent for the interface and injects the submitting end agent into a variable (such as a service variable in work) through spring. Here, instead of submitting the method-transmitted variables directly as a message, one layer is wrapped. So that variables required for management can be added to the submitted message. The final submitted message structure is: { message header: { requested, message creation time, method name signature, message encryption method }, message body: "message ciphertext". In the process of generating the submitting terminal agent by the specific sub-factory, the context variable called currently is obtained, the variable transmitted by the current calling method is obtained, the method variable is encrypted, all the variables are packaged, the specific submitting terminal client is called, and the packaged message is extracted into a message queue;
referring to fig. 4, a schematic diagram of the starter structure and process is shown. The internal structure of the initiator for acquiring and processing the message is shown in the figure, wherein the specific initiator needs to write each message queue well in advance. And (4) specifically determining which message queue is in the project, directly adding corresponding configuration, and starting the project. The initiator will invoke the message acquisition client to acquire and process the message. It can be seen that the developer's specific business implementation would be contained in the initiator. If the initiator is exiting unexpectedly (e.g., the machine is down) while the message is currently being processed. The currently processed message is not lost and will be processed again at the next start-up. Therefore, the problem that the messages are lost due to insufficient experience of developers is solved. When a starter for acquiring and processing the message works, after the message is acquired through a specific starter, namely a message acquisition client, reading head information from the message, storing the head information in a currently called context, and then decrypting method parameters according to encryption parameters in the head information; then calling a service interface to specifically realize processing service; judging whether normal processing is finished or not, if not, re-acquiring the current message and processing again, otherwise, removing the current message from the message queue by the message acquisition client and acquiring the next message;
when the message queue agent device is actually applied, only the required label is added on the interface, and the corresponding configuration is added. Taking Worker as an example, the method comprises the following steps: labeling at the interface:
@SenderFactory("kafkaSenderFactory")
publicinterface HelloService {
@MqTopic("sayHello")
publicvoid sayHello(String param1,String param2,String param3);
@MqTopic("sayworld")
publicvoid sayworld(String paramX,String paramY,String paramZ);
}
submitting message side agent factory configuration:
<bean class="cn.qtone.zf.common.mq.MqProduceConfig">
<property name="producers">
<list>
<value>HelloService</value>
</list>
</property>
</bean>
initiator configuration for message acquisition and processing
<bean class="cn.qtone.zf.common.mq.MqReceiverBootConfig">
<property name="receiverBoot" ref="KafkaReceiverBoot"/>
<property name="impls">
<map>
<entry key="org.xft.comon.HelloService" value-ref="helloServiceImpl"/>
</map>
</property>
</bean>
Therefore, by applying the message queue agent device in software development, a general developer can concentrate on business without writing the calling code of the message queue when using the message queue. This reduces the use and learning costs for the developer. Meanwhile, the difference of use caused by different personnel levels is eliminated, meanwhile, a complete log can be stored, and the message can be better ensured not to be lost due to downtime. The message queue service used can be switched at any time with zero impact on the traffic code.
Claims (9)
1. A message queue agent device is characterized by comprising a sending end agent, a receiving end agent and an agent internal protocol, wherein the sending end agent comprises a sending end agent interface and a plurality of sending end adapters; for each message queue in the agent internal protocol, at least one sending-end adapter and at least one receiving-end processor in the sending-end agent and the receiving-end agent are associated with each other to interact or communicate.
2. The message queue proxy device of claim 1, wherein the sending-end adapter is capable of sending the message obtained from the sending-end proxy interface to the corresponding topic of the message queue corresponding to the proxy internal protocol; the receiving end processor can be taken down from a message topic master-slave message queue to be processed by the receiving end agent interface, and then the receiving end agent interface is called to process specific services.
3. The message queue proxy device of claim 2, wherein the sender proxy gets a message from the sender proxy interface, encapsulates the message, and sends the encapsulated message; the receiving end proxy device needs to analyze the packaged message and then call the receiving end proxy device interface to process the specific service.
4. The message queue proxy device of claim 1, wherein the proxy internal protocols comprise any two or more of ActiveMQ, RabbitMQ, ZeroMQ, Kafka, MetaMQ, rocktmq, or Redis.
5. The message queue agent device according to any one of claims 1 to 4, comprising a sending end agent and a receiving end agent, including a submitting message end agent factory and a message acquiring and processing initiator, wherein the submitting message end agent factory is capable of automatically generating a submitting message agent of a service interface, and the message acquiring and processing initiator is capable of initiating a specific client for acquiring a message and invoking a specific service implementation to process a service.
6. The message queue agent device of claim 5, wherein when the agent factory at the submitting message end can automatically generate the submitting message agent of the service interface, the method comprises obtaining the context variable currently called, obtaining the variable transmitted by the currently called method, encrypting the method variable, packaging all the variables, calling the specific client at the submitting end, and extracting the packaged message into the message queue.
7. The message queue proxy device according to claim 5, wherein when the initiator for acquiring and processing the message works, after acquiring the message by a specific initiator, namely a message acquisition client, the initiator reads header information from the message and stores the header information in a currently called context, and then decrypts a method parameter according to an encryption parameter in the header information; then calling a service interface to specifically realize processing service; and then judging whether normal processing is finished or not, if not, re-acquiring the current message and processing again, otherwise, removing the current message from the message queue by the message acquisition client and acquiring the next message.
8. The message queue proxy device of claim 1, wherein the sender proxy interface and the sender proxy interface are the same interface.
9. The message queue agent apparatus of claim 1, wherein the message queue agent apparatus, when applied, includes adding a tag to the sending-end agent interface or the receiving-end agent interface, and configuring the factory for submitting the message-end agent and the initiator for obtaining and processing the message.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210335428.6A CN114979259A (en) | 2022-03-31 | 2022-03-31 | Message queue agent device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210335428.6A CN114979259A (en) | 2022-03-31 | 2022-03-31 | Message queue agent device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114979259A true CN114979259A (en) | 2022-08-30 |
Family
ID=82975597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210335428.6A Pending CN114979259A (en) | 2022-03-31 | 2022-03-31 | Message queue agent device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979259A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116126563A (en) * | 2023-02-20 | 2023-05-16 | 北京神州云合数据科技发展有限公司 | Message processing method, device, equipment and medium based on event bus |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1456005A (en) * | 2000-07-07 | 2003-11-12 | 软线股份公司 | Messaging proxy system |
| CN101207568A (en) * | 2007-03-16 | 2008-06-25 | 中国科学技术大学 | Multi protocol adapter and method for multi business to implement adapting treatment |
| US20150097697A1 (en) * | 2013-10-03 | 2015-04-09 | Duke Energy Corporation | Methods of processing data corresponding to a device that corresponds to a gas, water, or electric grid, and related devices and computer program products |
| CN111294317A (en) * | 2018-12-07 | 2020-06-16 | 北京新媒传信科技有限公司 | Method and device for RCS to support multi-protocol access and electronic equipment |
| WO2021179156A1 (en) * | 2020-03-10 | 2021-09-16 | 深圳市欢太科技有限公司 | Message processing method, device and system, and server |
-
2022
- 2022-03-31 CN CN202210335428.6A patent/CN114979259A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1456005A (en) * | 2000-07-07 | 2003-11-12 | 软线股份公司 | Messaging proxy system |
| US6721779B1 (en) * | 2000-07-07 | 2004-04-13 | Softwired Ag | Messaging proxy system |
| CN1671144A (en) * | 2000-07-07 | 2005-09-21 | 软线股份公司 | Messaging proxy system |
| CN101207568A (en) * | 2007-03-16 | 2008-06-25 | 中国科学技术大学 | Multi protocol adapter and method for multi business to implement adapting treatment |
| US20150097697A1 (en) * | 2013-10-03 | 2015-04-09 | Duke Energy Corporation | Methods of processing data corresponding to a device that corresponds to a gas, water, or electric grid, and related devices and computer program products |
| CN111294317A (en) * | 2018-12-07 | 2020-06-16 | 北京新媒传信科技有限公司 | Method and device for RCS to support multi-protocol access and electronic equipment |
| WO2021179156A1 (en) * | 2020-03-10 | 2021-09-16 | 深圳市欢太科技有限公司 | Message processing method, device and system, and server |
Non-Patent Citations (1)
| Title |
|---|
| 王腾 等: "基于MQTT协议的服务器中间消息队列设计与实现", 电脑编程技巧与维护 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116126563A (en) * | 2023-02-20 | 2023-05-16 | 北京神州云合数据科技发展有限公司 | Message processing method, device, equipment and medium based on event bus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111083161A (en) | Data transmission processing method and device and Internet of things equipment | |
| CN113039763B (en) | NF Service Consumer Restart Detection Using Direct Signaling Between NFs | |
| CN108664395B (en) | Application program testing method, device, equipment and storage medium | |
| CN110139165B (en) | Method for realizing one port bearing multiple stream protocols by stream media reverse proxy service | |
| CN112615753B (en) | Link abnormity tracking method, first node, second node and link | |
| CN102779071A (en) | Method, device and system for calling software interface | |
| CN113254103A (en) | Application function implementation method and device and storage medium | |
| CN114979259A (en) | Message queue agent device | |
| CN113342503B (en) | Real-time progress feedback method, device, equipment and storage medium | |
| WO2024103943A1 (en) | Service processing method and apparatus, storage medium, and device | |
| CN102438048B (en) | Method and system for calling remote service from Internet | |
| CN112689020A (en) | Message transmission method, message middleware, electronic equipment and storage medium | |
| CN116383840A (en) | Device for providing security support and operating system supporting national security protocol | |
| CN116828044A (en) | Remote sensing transmission method and system for message queue based on data plane development suite | |
| CN102510398A (en) | Request concurrent processing method and device, and server | |
| CN115426403A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110008032B (en) | Communication mode realization method and electronic equipment | |
| CN111221764B (en) | Cross-link data transmission method and system | |
| CN110008033B (en) | Method for communicating with client and electronic equipment | |
| CN113032123B (en) | Thread scheduling method, system and related device of remote NPL running environment | |
| CN113542217A (en) | Service subscription system | |
| CN111988283A (en) | Data transmission method, system, device and computer readable storage medium | |
| CN112073536A (en) | Method for realizing safe data transmission and processing between networks incapable of direct inter-access | |
| CN117201577B (en) | Communication method and system of cross-platform API and SPI based on PISA | |
| CN112953937B (en) | Communication end-to-end safety communication system of electric power trusted computing platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |