CN115174515A - Message distribution method based on file persistence - Google Patents
Message distribution method based on file persistence Download PDFInfo
- Publication number
- CN115174515A CN115174515A CN202210795331.3A CN202210795331A CN115174515A CN 115174515 A CN115174515 A CN 115174515A CN 202210795331 A CN202210795331 A CN 202210795331A CN 115174515 A CN115174515 A CN 115174515A
- Authority
- CN
- China
- Prior art keywords
- message
- file
- persistence
- subscription
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000002688 persistence Effects 0.000 title claims abstract description 36
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000002085 persistent effect Effects 0.000 claims abstract description 4
- 230000000903 blocking effect Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 238000006243 chemical reaction Methods 0.000 abstract description 3
- 230000010365 information processing Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
- H04L67/306—User profiles
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a message distribution method based on file persistence, which belongs to the technical field of information processing and comprises the following steps: the message publisher pushes the message to be distributed through the access address and enters a distribution subject corresponding to the system; the publishing and subscribing engine is used for finishing file persistence, message copying, message pushing, retransmission, result callback and pushing record storage of the message; when the message is pushed, according to the subscription service configured by the theme, determining the message format, the message processing flow and the service address of the subscription service, and pushing the message to the designated address after the message is processed and converted according to the requirement; in the persistent file system, a publishing subscription engine writes messages into files, the subscription files are isolated according to an application system, the files are scanned by using a single thread in the file scanning process and then are processed in a multithreading file message processing mode, the method supports conversion of different message formats and completely solves the problem of message transmission between enterprise client systems.
Description
Technical Field
The invention relates to the technical field of information processing, in particular to a message distribution method based on file persistence.
Background
Nowadays, when a certain service system needs to send a data message to a plurality of different service systems, they often use a subscription publishing mode, the service system needs to publish the message on an intermediate medium, the service system publishing the message is a message publisher (i.e. a party generating the message), the plurality of different service systems need to receive the message from the intermediate medium, and is a message subscriber (i.e. a message consumer), most of the current intermediate media are message queues, and the message queues support message publishing and message subscription, such as RabbitMQ, recketMQ, kafka, and ActiveMq, and most of them implement the use of a jms specification and an amqp specification, etc. If the business system uses the above as a publish-subscribe intermediary medium, the connection technology of the queue needs to be adapted according to the used queue middleware, resulting in technology learning cost and system modification cost; in addition, the MQ queue has single function, can only perform message caching and distribution, cannot completely solve the actual problem, and needs to be matched with other functional systems for use. In addition, the pure technology middleware has high use difficulty and unfriendly management interface interaction, so a novel message distribution mode needs to be designed in order to reduce the use difficulty of a service system.
Disclosure of Invention
The present invention aims to provide a message distribution method based on file persistence to solve the problems proposed in the above background art.
In order to achieve the purpose, the invention provides the following technical scheme: a message distribution method based on file persistence comprises the following steps:
s1: the message publisher pushes the message to be distributed through the access address and enters a distribution subject corresponding to the system;
s2: the publishing and subscribing engine is used for finishing file persistence, message copying, message pushing, retransmission, result callback and pushing record storage of the message;
s3: when the message is pushed, determining the message format, the message processing flow and the service address of the subscription service according to the subscription service configured by the theme, and pushing the message to the designated address after processing and converting the message according to the requirement;
s4: in the persistent file system, a publishing subscription engine writes messages into files, and the subscription files are isolated according to application systems, so that different systems are not influenced;
s5: the file scanning process related to the step S4 is to scan the file using a single thread and then to perform file processing in a manner of processing file messages using multiple threads.
Preferably, the data structure of the distribution topic, the data structure of the publish-subscribe engine and the data structure of the subscription service configuration are all implemented by using Java language.
Preferably, the distribution topic component, the publish-subscribe engine component and the subscription service configuration component are loaded on a one-stop business configuration interface.
Preferably, the step S3 specifically includes: the method comprises the steps that a subscription system atomic service flow receives tasks, pushes messages to a subscriber of the atomic service and judges whether the subscriber is successfully pushed or not; if not, judging whether retransmission is needed, otherwise, judging whether callback is needed, if so, recovering the context, and sending again after the interval time; if not, judging whether callback is needed, if so, judging whether callback is needed; and sequentially calling back according to the calling-back configuration, and then storing the calling result and the context information into the ES database, if the calling-back is not needed, directly storing the calling result and the context information into the ES database, and ending the process.
Preferably, the step S4 specifically includes: a service side of the distribution system receives the message of the publisher and judges whether to start asynchronous processing; if so, judging whether to start persistence; if the persistence is not started, after the message is split, asynchronously triggering the atomic service process of the subscription system; after finishing, returning the specified message to the issuing party, and ending; if the persistence is started, splitting the message according to the subscription system, writing the message into a file for persistence, returning the specified message to the publisher after the persistence is finished, and ending the process.
Preferably, if the asynchronous processing is not started in step S4, after the message is split, the atomic service process of the subscription system is synchronously triggered, the atomic service return result is blocked and waited, the subscription system return results are merged, and finally the merged result is returned to the publisher, and the process is finished.
Compared with the prior art, the invention has the beneficial effects that: the invention stores the message into the local file in a lasting way, uses the thread to scan the file to read the message content, analyzes the message content, and distributes the message to the destination system, which can be used for message publishing and message subscribing, the method realizes the one-to-one distribution of the message, the one-to-many distribution of the message, and the many-to-many distribution of the message, and simultaneously supports the conversion of different message formats, completely solves the problem of message transmission among enterprise client systems, functionally provides a series of functions of message persistence, message publishing, message splitting, message format conversion, protocol adaptation, caching, retrying, callback notification, and the like, and perfects the defects of using mq queue in the traditional message distribution scene; and a one-stop service configuration interface is provided, so that users can pay more attention to solving the service problem of enterprise message transmission, and the cost of the learning technology is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a flow chart of the subscription system service process of the present invention;
fig. 3 is a flow chart of message receiving processing of the distribution system of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention.
Referring to fig. 1, the present invention provides a technical solution: a message distribution method based on file persistence comprises the following steps:
s1: the message publisher pushes the message to be distributed through the access address and enters a distribution subject corresponding to the system;
s2: and finishing file persistence, message copying, message pushing, retransmission, result callback and pushing record storage of the message by using a publish-subscribe engine, wherein the message persistence: the information is ensured not to be lost, and the information persistence adopts a completely self-developed file storage module, so that the data is ensured not to be lost completely;
s3: when the message is pushed, according to the subscription service configured by the theme, determining the message format, the message processing flow and the service address of the subscription service, and pushing the message to the designated address after the message is processed and converted according to the requirement;
s4: in a persistent file system, a publishing subscription engine writes messages into files, and the subscription files are isolated according to application systems, so that different systems are not influenced by each other;
s5: the file scanning process related to the step S4 is to scan the file by using a single thread, and then to process the file by using a multithread file message processing method, so that the file processing efficiency is ensured.
The data structure of the distribution theme, the data structure of the publish-subscribe engine and the data structure of the subscription service configuration are all realized by using Java language, are used in a cross-platform manner and are independent of a third-party system, and the distribution theme component, the publish-subscribe engine component and the subscription service configuration component are loaded on a one-stop business configuration interface; the adaptive technology of the protocol solves the problem that each system of an enterprise sends or receives messages by using different protocols, and the related distribution priority is that a user can independently configure distribution resources, namely an independent message distribution thread pool, according to the importance degree of a service system, so that the systems are not influenced by each other.
Referring to fig. 2, step S3 specifically includes: the subscription system atomic service process receives the task, pushes the message to the subscriber of the atomic service, and judges whether the pushing of the subscriber is successful; if not, judging whether retransmission is needed, otherwise, judging whether callback is needed, if so, recovering the context, and sending again after the interval time; if not, judging whether callback is needed, if so, judging whether callback is needed; sequentially calling back according to the calling-back configuration, then storing the calling result and the context information into an ES database, if the calling-back is not needed, directly storing the calling result and the context information into the ES database, and ending the process at the same time, wherein the message retry specifically comprises the following steps: the message sending failure can be manually retried and automatically retried, the automatic retry can configure the maximum retry number and the interval time, and the manual retry carries out the backtracking retransmission according to the distribution log result.
Referring to fig. 3, step S4 specifically includes: the server side of the distribution system receives the message of the publisher and judges whether to start asynchronous processing; if yes, judging whether to start persistence; if the persistence is not started, after the message is split, asynchronously triggering the atomic service process of the subscription system; after finishing, returning the specified message to the issuing party, and ending; if the persistence is started, splitting the message according to a subscription system, wherein the message splitting is to split the message into a plurality of parts according to a subscribed service system, write the message into a file for persistence, and after the persistence is completed, returning the specified message to the publisher, and ending the same;
if the asynchronous processing is not started in the step S4, after the message is split, synchronously triggering the atomic service flow of the subscription system, blocking the returned result of the atomic service waiting, merging the returned result of the subscription system, and finally returning the merged result to the publisher, and ending.
The method does not need the technical transformation of the service system, and can be adapted to various service systems to complete message transmission; independent resources can be configured for a key service system, files, threads and protocols are persisted, and key services are guaranteed to be stable; the message can be subjected to service processing in the message forwarding process without developing a message processing module by a service system; the data is ensured to be accessible and automatic retransmission and manual retransmission are supported, and the message distribution records can be checked on an interface; callback notification of message distribution results is supported, and the callback notification can be known in time when the messages cannot be distributed due to network and other reasons; the function use bias business has low technical requirements on users, and the one-stop business configuration interface can adopt web interface configuration, so that the method is more convenient.
One case of using the message distribution method based on file persistence of the present invention: an enterprise has been developed through information for many years, and a plurality of business systems are provided in the enterprise, for example: when a certain employee enters an enterprise, the information of the employee needs to be input into each business system of the enterprise, so that business users can input the employee information for many times, the time cost of inputting the information is increased, the error rate of information inputting is increased, the information consistency is not guaranteed, if the information distribution system is used, the HR is used as an input entry of the employee information, the HR publishes the input employee information onto the distribution system, and the distribution system forwards the input employee information to the OA, the CRM and the financial system, so that the HR only needs to input the employee information once, other business systems can receive the employee information as message subscribers, the time cost of inputting the information is reduced, the labor cost is reduced, the error rate of information inputting is reduced, and the information consistency is guaranteed.
In the description herein, references to the description of "one embodiment," "an example," "a specific example," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (6)
1. A message distribution method based on file persistence is characterized by comprising the following steps:
s1: the message publisher pushes the message to be distributed through the access address and enters a distribution subject corresponding to the system;
s2: the publishing and subscribing engine is used for finishing the file persistence of the message, message copying, message pushing, retransmission, result callback and pushing record warehousing;
s3: when the message is pushed, according to the subscription service configured by the theme, determining the message format, the message processing flow and the service address of the subscription service, and pushing the message to the designated address after the message is processed and converted according to the requirement;
s4: in a persistent file system, a publishing subscription engine writes messages into files, and the subscription files are isolated according to application systems, so that different systems are not influenced by each other;
s5: the file scanning process related to the step S4 is to scan the file using a single thread and then to perform file processing in a manner of processing file messages using multiple threads.
2. The message distribution method based on file persistence of claim 1, wherein: the data structure of the distribution topic, the data structure of the publish-subscribe engine and the data structure of the subscription service configuration are all implemented by using Java language.
3. The message distribution method based on file persistence according to claim 2, wherein: the distribution topic component, the publish-subscribe engine component and the subscribe service configuration component are loaded on a one-stop business configuration interface.
4. The message distribution method based on file persistence of claim 1, wherein: the step S3 specifically comprises the following steps: the method comprises the steps that a subscription system atomic service flow receives tasks, pushes messages to a subscriber of the atomic service and judges whether the subscriber is successfully pushed or not; if not, judging whether retransmission is needed, otherwise, judging whether callback is needed, if so, recovering the context, and sending again after the interval time; if not, judging whether callback is needed, if so, judging whether callback is needed; and sequentially calling back according to the calling-back configuration, and then storing the calling result and the context information into the ES database, if the calling-back is not needed, directly storing the calling result and the context information into the ES database, and ending the process.
5. The message distribution method based on file persistence according to claim 1, wherein: the step S4 specifically includes: a service side of the distribution system receives the message of the publisher and judges whether to start asynchronous processing; if yes, judging whether to start persistence; if persistence is not started, after the message is split, asynchronously triggering a subscription system atomic service flow; after finishing, returning the specified message to the issuing party, and ending; if the persistence is started, splitting the message according to the subscription system, writing the message into a file for persistence, returning the specified message to the publisher after the persistence is finished, and ending the process.
6. The message distribution method based on file persistence according to claim 5, wherein: if the asynchronous processing is not started in the step S4, after the message is split, synchronously triggering the atomic service flow of the subscription system, blocking the return result of the atomic service waiting, merging the return result of the subscription system, and finally returning the merged result to the publisher, and ending.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210795331.3A CN115174515A (en) | 2022-07-07 | 2022-07-07 | Message distribution method based on file persistence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210795331.3A CN115174515A (en) | 2022-07-07 | 2022-07-07 | Message distribution method based on file persistence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115174515A true CN115174515A (en) | 2022-10-11 |
Family
ID=83491459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210795331.3A Pending CN115174515A (en) | 2022-07-07 | 2022-07-07 | Message distribution method based on file persistence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115174515A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090125420A1 (en) * | 2007-11-14 | 2009-05-14 | Xuefu Zhang | Web-based information delivery method, system, and apparatus |
| CN102562162A (en) * | 2011-12-14 | 2012-07-11 | 北京邮电大学 | Event-handling-based coal mine alarm system and method |
| CN104954469A (en) * | 2015-06-19 | 2015-09-30 | 长沙廖氏软件科技有限公司 | Information exchange method for heterogeneous system |
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN109491617A (en) * | 2018-11-16 | 2019-03-19 | 广州创想云科技有限公司 | Data storage retransmission method, electronic equipment, storage medium |
| CN110941622A (en) * | 2019-10-31 | 2020-03-31 | 京东数字科技控股有限公司 | Data processing method and device |
| CN113157449A (en) * | 2021-04-16 | 2021-07-23 | 上海寰果信息科技有限公司 | Real-time stream data analysis processing method based on MQTT |
| CN114116632A (en) * | 2021-11-22 | 2022-03-01 | 中国建设银行股份有限公司 | Transaction log collection method, device and system |
-
2022
- 2022-07-07 CN CN202210795331.3A patent/CN115174515A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090125420A1 (en) * | 2007-11-14 | 2009-05-14 | Xuefu Zhang | Web-based information delivery method, system, and apparatus |
| CN102562162A (en) * | 2011-12-14 | 2012-07-11 | 北京邮电大学 | Event-handling-based coal mine alarm system and method |
| CN104954469A (en) * | 2015-06-19 | 2015-09-30 | 长沙廖氏软件科技有限公司 | Information exchange method for heterogeneous system |
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN109491617A (en) * | 2018-11-16 | 2019-03-19 | 广州创想云科技有限公司 | Data storage retransmission method, electronic equipment, storage medium |
| CN110941622A (en) * | 2019-10-31 | 2020-03-31 | 京东数字科技控股有限公司 | Data processing method and device |
| CN113157449A (en) * | 2021-04-16 | 2021-07-23 | 上海寰果信息科技有限公司 | Real-time stream data analysis processing method based on MQTT |
| CN114116632A (en) * | 2021-11-22 | 2022-03-01 | 中国建设银行股份有限公司 | Transaction log collection method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1360561B1 (en) | System and method for bounding the life of an event subscription to the availability of an object | |
| US8407297B2 (en) | Systems and methods to receive information from a groupware client | |
| US7216142B2 (en) | Network application program interface facilitating communication in a distributed network environment | |
| US7680940B2 (en) | Method and system for managing dynamic associations between folksonomic data and resources | |
| US6910070B1 (en) | Methods and systems for asynchronous notification of database events | |
| CN112527525B (en) | Distributed event bus processing method, terminal and medium based on message queue | |
| US7840611B2 (en) | High availability for event forwarding | |
| CN100596050C (en) | Message reliable informing method and system between systems | |
| CN111371892A (en) | High-concurrency distributed message pushing system and method | |
| US20080098406A1 (en) | Delivering certified messages to message servers | |
| US20090106371A1 (en) | Systems and methods to generate business reports based on electronic mail messages | |
| US7640263B2 (en) | Queued system event notification and maintenance | |
| US20030002634A1 (en) | Event notification in a unified message system using an event notification server | |
| US7315863B2 (en) | Method and apparatus for automatic notification of database events | |
| US7836123B2 (en) | System and method for non-HTTP session based publish/subscribe support using pre-emptive subscriptions | |
| CN114363407B (en) | Message service method and device, readable storage medium and electronic equipment | |
| US20090106372A1 (en) | Systems and methods to transmit information to a groupware client | |
| WO2021088641A1 (en) | Data transmission method, data processing method, data reception method and device, and storage medium | |
| CN110968586A (en) | Distributed transaction processing method and device | |
| CA2725017A1 (en) | Email auto-filing and management | |
| CN114063936B (en) | Method, system, equipment and storage medium for optimizing timing task | |
| CN110727507B (en) | Message processing method and device, computer equipment and storage medium | |
| CN114629866A (en) | Method for establishing real-time communication between multiple systems by random online customer service | |
| CN115174515A (en) | Message distribution method based on file persistence | |
| CN114443325A (en) | Asynchronous retry method and system for interface call exception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |