CN108833443A - A kind of method for message transmission and system, computer equipment - Google Patents
A kind of method for message transmission and system, computer equipment Download PDFInfo
- Publication number
- CN108833443A CN108833443A CN201810833542.5A CN201810833542A CN108833443A CN 108833443 A CN108833443 A CN 108833443A CN 201810833542 A CN201810833542 A CN 201810833542A CN 108833443 A CN108833443 A CN 108833443A
- Authority
- CN
- China
- Prior art keywords
- message
- configuration
- safe handling
- consumer
- certificate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000005540 biological transmission Effects 0.000 title claims abstract description 38
- 238000003860 storage Methods 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 abstract description 12
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000011161 development Methods 0.000 abstract description 6
- 238000012423 maintenance Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 13
- 230000009286 beneficial effect Effects 0.000 description 7
- 230000018109 developmental process Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000002716 delivery method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0823—Network architectures or network communication protocols for network security for authentication of entities using certificates
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/16—Implementing security features at a particular protocol layer
- H04L63/168—Implementing security features at a particular protocol layer above the transport layer
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The present invention relates to a kind of field of computer technology more particularly to a kind of method for message transmission and system, computer equipment, method to include：After program starting, message system loads security configuration table, and when configuration data, which is, needs safe handling, obtains configuration information from security configuration table；Receive the message by safe handling that there is the message producer of certificate to send；Message is encrypted according to configuration information, so that the message consumer with certificate receives encrypted message.The medium that the present invention is transmitted by using message system as message, support distributed deployment, message efficiency of transmission greatly improves, and can reduce the development difficulty and maintenance cost of system after new demand is added in a manner of flexible configuration safe handling and safety verification, message format and method of calibration.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of method for message transmission and systems, computer equipment.
Background technique
Webservice service or IBM are generallyd use generally directed to the message delivery method between heterogeneous system and system
MQ message queue technology completes data interaction.Using transmission both sides when webservice technology according to the interface specification document of agreement
Service publication and request call are completed, mutual data interaction can be completed.When using IBM MQ message queue technology, MQ
Product carries out message transmission as the important intermediary of message transmission.The product provides the API of a variety of development languages.Transmission
The specific API that both sides need to provide according to MQ carries out message and sends and receives.
Disappear however, being existed in high concurrent scene using the messaging system of webservice technology and IBM MQ technology
Breath transmitting and the low problem for the treatment of effeciency.
Summary of the invention
The technical problem to be solved by the present invention is in view of the deficiencies of the prior art, provide a kind of method for message transmission
And system, computer equipment, it is able to ascend the efficiency of message transmission and processing in high concurrent scene.
The technical solution that the present invention solves above-mentioned technical problem is as follows：A kind of method for message transmission, including：
After program starting, message system loads security configuration table, and when configuration data, which is, needs safe handling, from the peace
Configuration information is obtained in full allocation list, wherein the security configuration table includes：Whether message system safe handling, message are carried out
Whether system safe handling mode the configuration of message system certificate position, carries out message safe handling, message password；
Receive the message by safe handling that there is the message producer of certificate to send；
The message is encrypted according to the configuration information, is added to be received with the message consumer of the certificate
The message after close.
The beneficial effects of the invention are as follows：By using the medium that message system is transmitted as message, distributed deployment is supported,
Message efficiency of transmission greatly improves, can be in a manner of flexible configuration safe handling and safety verification, message format and method of calibration, drop
The development difficulty and maintenance cost of system after low new demand is added.
Based on the above technical solution, the present invention can also be improved as follows.
Further, further include：The message producer is according to the configuration information by data conversion to be sent at predetermined
The message of format, and the message is sent to the message system.
Further, further include：
The message consumer verifies the received message according to checking prediction message, and the checking prediction message is configuration
The message with the predetermined format；
When check results are not pass through, handled according to the failure handling mode of configuration.
Beneficial effect using above-mentioned further scheme is：By using configurable safe handling and safety verification,
Configurable message format conversion and verification, sender of the message and recipient are flexibly matched with, messaging system is reached
Maximum flexibility.And it is suitable for the message transmission under big data quantity, high concurrent scene, when newly-increased message sender and message connect
When debit, it is only necessary to which new business number can be transmitted by adjusting message format, security strategy and the configuration of kafka message system, system
According to, considerably reduce because demand change caused by a large amount of developments.
Further, the predetermined format includes：Type of message, message categories, message label, fixed value, message format
With failure handling mode, the method also includes：
In the buffer by the predetermined format and checking prediction message storage.
Beneficial effect using above-mentioned further scheme is：By the way that the configuration informations such as predetermined format and checking prediction message are stored
In local cache, avoids reading from database every time, can effectively promote the speed of message transmission.
Further, the message producer and the message consumer are multiple, the method also includes：
The message system carries out kafka subject classification according to message categories；
Wherein, the message by safe handling for receiving the message producer with certificate and sending, including：
The message that the message system receives multiple message producers while sending, and by each message
Partitioned storage is carried out in the theme corresponding to its corresponding message categories；
Each message consumer subscribes to the messages of multiple message categories simultaneously, and from the corresponding master of each message categories
Corresponding message is read in the subregion of topic.
Beneficial effect using above-mentioned further scheme is：By message categories partitioned storage mode, solves high concurrent
The bottleneck that message is transmitted in scene, message consumer can be processed in batches the message in queue, processing speed faster, and message
It can be by multiple consumer's Double Spendings.
Further, the method also includes：
When the message consumer receives message failure, reception is retried；
When the message consumer retries number of retries of the received number beyond configuration, send the messages to pair
It answers in the error message theme of theme, and record log information.
Beneficial effect using above-mentioned further scheme is：It, can be with the reliability of lifting system by using strategy is retried.
Further, the configuration information includes：Security protocol, certificate position and password configuration field, wherein the peace
Full processing mode is determined that the security protocol includes by the security protocol：Ssl protocol, SASL_PLAINTEXT agreement and
SASL_SSL agreement.
Further, the method also includes：The message received is saved in memory number by the message consumer
According in library, and the expired time of the message is set.
Beneficial effect using above-mentioned further scheme is：By using the strategy that message file backs up, it can be improved and be
The reliability of system.
The another technical solution that the present invention solves above-mentioned technical problem is as follows：A kind of computer equipment, including memory,
Processor and storage on a memory and the computer program that can run on a processor, when the processor execution described program
The step of realizing the method for message transmission in such as above-described embodiment.
The another technical solution that the present invention solves above-mentioned technical problem is as follows：A kind of message delivery system, including：It is multiple
The message producer, message system and multiple message consumers, wherein
The message system, for after program starting, loading security configuration table, and when configuration data is to need safe handling
When, configuration information is obtained from the security configuration table, wherein the security configuration table includes：Whether message system peace is carried out
Whether full processing, message system safe handling mode, the configuration of message system certificate position and password configuration carry out message safe place
Reason, message password；
Each message producer, for according to the configuration information by data conversion to be sent at the report of predetermined format
Text, and the message is sent to the message system；
The message system is also used to receive the report by safe handling that there is the message producer of certificate to send
Text, and the message is encrypted according to the configuration information；
Each message consumer, for being verified according to checking prediction message to the received message, the verification
Message is the message with the predetermined format of configuration, and when check results are not pass through, according to the failure handling of configuration
Mode is handled.
The beneficial effects of the invention are as follows：Message delivery system quotes kafka message components, passes through distributed, more points of support
The kafka message system in area, solves the bottleneck that message is transmitted in high concurrent scene, and message consumer can be processed in batches queue
In message, processing speed is faster.And this system uses configurable safe handling and safety verification, configurable message
Format conversion and verification, sender of the message and recipient are flexibly matched with, the maximum flexibility of messaging system is reached.
The advantages of additional aspect of the invention, will be set forth in part in the description, and will partially become from the following description
It obtains obviously, or practice is recognized through the invention.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to the embodiment of the present invention or the prior art
Attached drawing needed in description is briefly described, it should be apparent that, drawings described below is only of the invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other attached drawings.
Fig. 1 is a kind of schematic flow chart of method for message transmission described in the embodiment of the present invention；
Fig. 2 is a kind of schematic flow chart of method for message transmission described in another embodiment of the present invention；
Fig. 3 is a kind of schematic flow chart of method for message transmission described in another embodiment of the present invention；
Fig. 4 is a kind of schematic flow chart of method for message transmission described in another embodiment of the present invention；
Fig. 5 is a kind of schematic flow chart of method for message transmission described in another embodiment of the present invention；
Fig. 6 is a kind of schematic flow chart of method for message transmission described in another embodiment of the present invention；
Fig. 7 is a kind of schematic block diagram of message delivery system described in the embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiment is a part of the embodiments of the present invention, rather than whole embodiments.Based on this hair
Embodiment in bright, those of ordinary skill in the art's every other reality obtained without making creative work
Example is applied, all should belong to the scope of protection of the invention.
As shown in Figure 1, a kind of method for message transmission 100 includes：
101, after program starting, message system loads security configuration table, and when configuration data, which is, needs safe handling, from peace
Configuration information is obtained in full allocation list.
Wherein, security configuration table may include：Whether message system safe handling, message system safe handling side are carried out
Whether formula the configuration of message system certificate position, carries out message safe handling, message password.
102, the message by safe handling that there is the message producer of certificate to send is received.
103, message is encrypted according to configuration information, so that the message consumer with certificate receives encrypted report
Text.
Specifically, in this embodiment, after program starting, the whole security configuration table of message system load, when being used to match
The configuration data for setting message system is when needing safe handling, to obtain configuration information from security configuration table.Wherein, security configuration table
In include whether carry out message system safe handling, message system safe handling mode, message system certificate position configuration, be
The contents such as no progress message safe handling, message password.
Configuration data is the allocation list information in software database, for configuring system；Message and message are then software
Information to be processed.
The message producer and message consumer are separation.The message producer needs to configure message in security configuration table raw
The certification path of production person；Message consumer needs to configure the certification path of message consumer in security configuration table.Certificate content
Including：Username and password.
It should be noted that in this embodiment, configuration information may include：Security protocol, certificate position and password are matched
Set field etc., it should be noted that configuration information may include more parameters, and the embodiment of the present invention is not limited to and this.
Wherein, safe handling mode is determined by security protocol, such as：Security protocol may include：Ssl protocol, SASL_
PLAINTEXT agreement and SASL_SSL agreement, the embodiment of the present invention are not limited in any way this.
It should be understood that in this embodiment, message system can use kafka message components.
A kind of method for message transmission provided in above-described embodiment, the matchmaker transmitted by using message system as message
It is situated between, supports distributed deployment, message efficiency of transmission greatly improves, can be in a manner of flexible configuration safe handling and safety verification, report
Literary format and method of calibration reduce the development difficulty and maintenance cost of system after new demand is added.
Optionally, in one embodiment, as shown in Fig. 2, before 102, this method 100 can also include：
104, the message producer according to configuration information by data conversion to be sent at the message of predetermined format, and to message system
System sends message.
Specifically, in this embodiment, after program starting, message system loads security configuration table, and when configuration data is
When needing safe handling, configuration information is obtained from security configuration table.The message producer with certificate is matched from message system
Confidence breath, and data conversion to be sent is sent to message system at after the message of predetermined format by the message according to configuration information.
Message system receives the message, and is encrypted according to configuration information to the message, so as to the message consumer with the certificate
Receive the encrypted message.
It should be understood that in this embodiment, same a collection of data to be transmitted can be converted into predetermined format by the message producer
Message is sent after automatic compression to message system, and message consumer can receive and decompress in batches, thus realize batch processing,
Reduce network load.
Optionally, in another embodiment, as shown in figure 3, after 103, this method 100 can also include：
105, message consumer verifies received message according to checking prediction message, and checking prediction message is that having for configuration is pre-
The message for the formula that fixes.
106, it when check results are not pass through, is handled according to the failure handling mode of configuration.
Specifically, in this embodiment, after program starting, message system loads security configuration table, and when configuration data is
When needing safe handling, configuration information is obtained from security configuration table.The message producer with certificate is matched from message system
Confidence breath, and data conversion to be sent is sent to message system at after the message of predetermined format by the message according to configuration information.
Message system receives the message, and is encrypted according to configuration information to the message, and the message consumer with the certificate receives
After the encrypted message, received message is verified according to checking prediction message, checking prediction message is that having for configuration fixes in advance
The message of formula.
It should be noted that the verification to the verification of message specially to message format here, such as：Whether message label name
Correctly, quantity is consistent etc..
When check results are not pass through, handled according to the failure handling mode of configuration.Such as：Record log or throwing
Abandon etc..When check results be by when, message is subjected to corresponding business processing, data are stored in database.
A kind of method for message transmission provided in above-described embodiment is tested by using configurable safe handling and safety
Card, configurable message format conversion and verification, sender of the message and recipient are flexibly matched with, message transmission system is reached
The maximum flexibility of system.And it is suitable for the message transmission under big data quantity, high concurrent scene, when newly-increased message sender and message
When recipient, it is only necessary to which new business can be transmitted by adjusting message format, security strategy and the configuration of kafka message system, system
Data considerably reduce a large amount of developments because of caused by demand change.
Optionally, in another embodiment, predetermined format includes：Type of message, message categories, message label, fixation
Value, message format and failure handling mode, this method 100 can also include：
In the buffer by predetermined format and checking prediction message storage.
In this embodiment, it by the way that the configuration informations such as predetermined format and checking prediction message are stored in local cache, avoids
It is read every time from database, can effectively promote the speed of message transmission.
Optionally, in another embodiment, as shown in figure 4, the message producer and message consumer are multiple, the party
Method 100 can also include：
107, message system carries out kafka subject classification according to message categories.
Then, step 104 is specially：
Multiple message producers according to configuration information by respective data conversion to be sent at the message of predetermined format, and it is same
When to message system send message.
Then, step 102 is specially：
The message that message system receives multiple message producers while sending, and by each message in its corresponding message kind
Partitioned storage is carried out in theme corresponding to class.
108, each message consumer subscribes to the messages of multiple message categories simultaneously, and from the corresponding master of each message categories
Corresponding message is read in the subregion of topic.
Specifically, in this embodiment, the message producer and message consumer be it is multiple, message system can be according to disappearing
It ceases type and carries out kafka subject classification, when multiple message producers send message to message system simultaneously, message system will be connect
The each message received carries out partitioned storage, a theme corresponding to the message categories of each message, and different themes are stored in difference
In subregion.And each message consumer can subscribe to the message of multiple message categories simultaneously, and corresponding from each message categories
Corresponding message is read in the subregion of theme.
Such as：The message producer is 6, and message consumer is 2, and message categories have 5 kinds, then kafka theme is corresponding with 5
A classification.6 message producers send message to message system simultaneously, are denoted as respectively：Message 1, message 2, message 3, message 4,
Message 5 and message 6.Message 1, message 2, message 3 are same kind of message；Message 4, message 5 are same kind of message；Report
Text 6 is the message of a type.
Received each message is carried out partitioned storage, a master corresponding to the message categories of each message by message system
Topic, different themes are stored in different subregions.That is：Message 1, message 2, message 3 are stored in its theme A corresponding one
In a subregion A；Message 4, message 5 are stored in the corresponding subregion B of its theme B；It is corresponding that message 6 is stored in its theme C
In one subregion C.
And each message consumer can subscribe to the message of multiple message categories simultaneously, and corresponding from each message categories
Corresponding message is read in the subregion of theme.Such as：Message consumer 1 has subscribed 2 message categories simultaneously, corresponding theme A and
Theme C；Message consumer 2 has subscribed 2 message categories, corresponding theme B and theme C simultaneously.Then, message consumer 1 respectively from
Corresponding message is read in the corresponding subregion C of theme A corresponding subregion A and theme C；Message consumer 2 is corresponding from theme B respectively
Subregion B and the corresponding subregion C of theme C in read corresponding message.
A kind of method for message transmission provided in above-described embodiment solves height by message categories partitioned storage mode
The bottleneck of message transmission in concurrent scene, message consumer can be processed in batches the message in queue, processing speed faster, and
Message can be by multiple consumer's Double Spendings.
It should be noted that in this embodiment, the quantity of the message producer and message consumer can identical can also be with
Difference, the embodiment of the present invention are not limited in any way this.
It can also include step 104 perhaps step 109-110 or step 111 it should be understood that in this embodiment, or
The combination of person's above-mentioned steps, succinct for description, details are not described herein.
Optionally, in another embodiment, as shown in figure 5, this method 100 can also include：
109, when message consumer receives message failure, reception is retried.
110, when message consumer retries number of retries of the received number beyond configuration, corresponding master is sent by message
In the error message theme of topic, and record log information.
In order to solve the problems, such as that batch processing is retried processing obstruction, this programme is being sent and received (i.e.：Message generator and
Message consumer) between use and reliably retry strategy.Specifically, in this embodiment, being lost when message consumer receives message
When losing, reception can be retried, and when retrying number of retries of the received number beyond configuration, sends corresponding theme for message
Error message theme in, and record log information, message consumer can continue to next message.
It should be noted that in this embodiment, can also include step 104 perhaps step 107,104,108 or
Step 111 or the combination of above-mentioned steps, succinct for description, details are not described herein.
Optionally, in another embodiment, as shown in fig. 6, this method 100 can also include：
111, the message received is saved in memory database by message consumer, and be arranged the message it is expired when
Between.
In order to improve the reliability of system, the strategy of message file backup is additionally used in this programme.Specifically, in the reality
It applies in example, received message is saved in memory database by message consumer, and expired time is arranged.When there are abnormal conditions
When, it can be searched problem reason by checking the original message received in memory database.When reaching expired time, this is abandoned
Message, to store the message of receipt of subsequent using the memory headroom for vacating message occupancy.
It should be understood that in the above embodiments, the content of message and message be substantially it is identical, message is that message passes
One of defeated process concrete form.In addition, the message producer and message consumer in the various embodiments described above are to have card
Book, the transmission of message only can be just carried out by message system with certificate.
It should be noted that in this embodiment, can also include step 104 perhaps step 104,105-106 or
The combination of step 107-108 perhaps step 109-110 or above-mentioned steps, succinct for description, details are not described herein.
It should be understood that magnitude of the sequence numbers of the above procedures are not meant to execution sequence in various embodiments of the present invention
Successively, the execution sequence of each process should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention
Constitute any restriction.
The embodiment of the present invention also provides a kind of computer equipment, including memory, processor and storage are on a memory simultaneously
The computer program that can be run on a processor, processor are realized when executing described program such as the message transmission in above-described embodiment
The step of method.
A kind of technical solution of method for message transmission is described in detail above in conjunction with attached drawing 1-6,7 in detail with reference to the accompanying drawing
A kind of technical solution of message delivery system is carefully described.
A kind of message delivery system 200 as shown in Figure 7 includes：Multiple message producers 210, message system 220 and more
A message consumer 230, wherein
Message system 220, for after program starting, loading security configuration table, and when configuration data is to need safe handling
When, configuration information is obtained from security configuration table.Wherein, security configuration table includes：Whether carry out message system safe handling, disappear
Breath system safe handling mode, the configuration of message system certificate position and password configuration, whether to carry out message safe handling, message close
Code.
Each message producer 210, for according to configuration information by data conversion to be sent at the message of predetermined format, and
Message is sent to message system 220.
Message system 220 is also used to receive the report by safe handling that there is the message producer 210 of certificate to send
Text, and message is encrypted according to configuration information.
Each message consumer 230, for being verified according to checking prediction message to received message, checking prediction message is configuration
The message with predetermined format, and when check results be not by when, handled according to the failure handling mode of configuration.
Specifically, in this embodiment, each message producer 210 and each message consumer 230 may have disappearing
The certificate that breath system 220 configures, to ensure that message transmission can be carried out between both sides.In addition, each message producer 210 can be with
It will be converted into the message of predetermined format with batch of data to be transmitted, is sent after being compressed to message system 220, information consumption
Person 230 can receive and decompress in batches, to realize batch processing, reduce network load.
Message delivery system in above-described embodiment quotes kafka message components, by distributed, support multi partition
Kafka message system, solves message transmission bottleneck in high concurrent scene, and message consumer can be processed in batches disappearing in queue
Breath, processing speed is faster.And this system uses configurable safe handling and safety verification, configurable message format turns
It changes and verifies, sender of the message and recipient are flexibly matched with, reach the maximum flexibility of messaging system.
Optionally, in one embodiment, message system 220 can also carry out kafka subject classification according to message categories,
The message for receiving multiple message producers while sending, and by each message in the theme corresponding to its corresponding message categories
Carry out partitioned storage.Each message consumer 230 can subscribe to the message of multiple message categories simultaneously, and from each message categories
Corresponding message is read in the subregion of corresponding theme.
Optionally, in another embodiment, each message consumer 230 is also used to retry when receiving message failure
It receives, and when retrying number of retries of the received number beyond configuration, sends message to the error message master of corresponding theme
In topic, and record log information.
In addition, each message consumer 230, the message for being also used to receive is saved in memory database, and is arranged
The expired time of the message can be by checking the original report received in memory database so as to when there are abnormal conditions
Text searches problem reason.
In addition, the terms "and/or", only a kind of incidence relation for describing affiliated partner, indicates may exist
Three kinds of relationships, for example, A and/or B, can indicate：Individualism A exists simultaneously A and B, these three situations of individualism B.Separately
Outside, character "/" herein typicallys represent the relationship that forward-backward correlation object is a kind of "or".
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Specially
Industry technical staff can use different methods to achieve the described function each specific application, but this realization is not
It is considered as beyond the scope of this invention.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is
The specific work process of system, device and unit, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the division of unit,
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be with
In conjunction with or be desirably integrated into another system, or some features can be ignored or not executed.In addition, shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING of device or unit or logical
Letter connection is also possible to electricity, mechanical or other form connections.
Unit may or may not be physically separated as illustrated by the separation member, shown as a unit
Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks
On unit.It can select some or all of unit therein according to the actual needs to realize the mesh of the embodiment of the present invention
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
It, can if integrated unit is realized in the form of SFU software functional unit and when sold or used as an independent product
To be stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention substantially or
Say that all or part of the part that contributes to existing technology or the technical solution can embody in the form of software products
Out, which is stored in a storage medium, including some instructions are used so that a computer equipment
(can be personal computer, server or the network equipment etc.) executes all or part of each embodiment method of the present invention
Step.And storage medium above-mentioned includes：It is USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random
Access various Jie that can store program code such as memory (RAM, Random Access Memory), magnetic or disk
Matter.
More than, only a specific embodiment of the invention, but scope of protection of the present invention is not limited thereto, and it is any to be familiar with
Those skilled in the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or substitutions,
These modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be wanted with right
Subject to the protection scope asked.
Claims (10)
1. a kind of method for message transmission, which is characterized in that including：
After program starting, message system loads security configuration table, and when configuration data, which is, needs safe handling, matches from the safety
It sets and obtains configuration information in table, wherein the security configuration table includes：Whether message system safe handling, message system are carried out
Whether safe handling mode the configuration of message system certificate position, carries out message safe handling, message password；
Receive the message by safe handling that there is the message producer of certificate to send；
The message is encrypted according to the configuration information, after receiving encryption with the message consumer of the certificate
The message.
2. the method according to claim 1, wherein further including：
The message producer disappears data conversion to be sent at the message of predetermined format, and to described according to the configuration information
Breath system sends the message.
3. according to the method described in claim 2, it is characterized in that, further including：
The message consumer verifies the received message according to checking prediction message, and the checking prediction message is the tool of configuration
There is the message of the predetermined format；
When check results are not pass through, handled according to the failure handling mode of configuration.
4. according to the method described in claim 3, it is characterized in that, the predetermined format includes：Type of message, message categories,
Message label, fixed value, message format and failure handling mode, the method also includes：
In the buffer by the predetermined format and checking prediction message storage.
5. according to the method any in claim 2-4, which is characterized in that the message producer and the information consumption
Person be it is multiple, the method also includes：
The message system carries out kafka subject classification according to message categories；
Wherein, the message by safe handling for receiving the message producer with certificate and sending, including：
The message that the message system receives multiple message producers while sending, and by each message at it
Partitioned storage is carried out in theme corresponding to corresponding message categories；
Each message consumer subscribes to the messages of multiple message categories simultaneously, and from the corresponding theme of each message categories
Corresponding message is read in subregion.
6. according to the method described in claim 5, it is characterized in that, the method also includes：
When the message consumer receives message failure, reception is retried；
When the message consumer retries number of retries of the received number beyond configuration, corresponding master is sent the messages to
In the error message theme of topic, and record log information.
7. method according to any of claims 1-4, which is characterized in that the configuration information includes：Security protocol,
Certificate position and password configuration field, wherein the safe handling mode determines by the security protocol, the security protocol packet
It includes：Ssl protocol, SASL_PLAINTEXT agreement and SASL_SSL agreement.
8. method according to any of claims 1-4, which is characterized in that the method also includes：
The message received is saved in memory database by the message consumer, and be arranged the message it is expired when
Between.
9. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor realizes method for message transmission described in claim 1-8 when executing described program
The step of.
10. a kind of message delivery system, which is characterized in that including：Multiple message producers, message system and multiple information consumptions
Person, wherein
The message system, for loading security configuration table, and when configuration data, which is, needs safe handling after program starting,
Configuration information is obtained from the security configuration table, wherein the security configuration table includes：Whether message system safe place is carried out
Reason, message system safe handling mode, message system certificate position configuration and password configuration, whether carry out message safe handling,
Message password；
Each message producer, for according to the configuration information by data conversion to be sent at the message of predetermined format,
And the message is sent to the message system；
The message system is also used to receive the message by safe handling that there is the message producer of certificate to send,
And the message is encrypted according to the configuration information；
Each message consumer, for being verified according to checking prediction message to the received message, the checking prediction message
For the message with the predetermined format of configuration, and when check results be not by when, according to the failure handling mode of configuration
It is handled.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810833542.5A CN108833443B (en) | 2018-07-26 | 2018-07-26 | Message transmission method and system and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810833542.5A CN108833443B (en) | 2018-07-26 | 2018-07-26 | Message transmission method and system and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108833443A true CN108833443A (en) | 2018-11-16 |
| CN108833443B CN108833443B (en) | 2021-02-02 |
Family
ID=64140058
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810833542.5A Active CN108833443B (en) | 2018-07-26 | 2018-07-26 | Message transmission method and system and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108833443B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110806958A (en) * | 2019-10-24 | 2020-02-18 | 长城计算机软件与系统有限公司 | Monitoring method, monitoring device, storage medium and electronic equipment |
| CN110995851A (en) * | 2019-12-11 | 2020-04-10 | 贝壳技术有限公司 | Message processing method, device, storage medium and equipment |
| CN111262915A (en) * | 2020-01-10 | 2020-06-09 | 北京东方金信科技有限公司 | Kafka cluster-crossing data conversion system and method |
| CN111866189A (en) * | 2020-09-03 | 2020-10-30 | 中国银行股份有限公司 | Batch file transmission method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080083026A1 (en) * | 2006-10-02 | 2008-04-03 | Bea Systems, Inc. | Kerberos Protocol Security Provider for a Java Based Application Server |
| CN104869174A (en) * | 2015-06-15 | 2015-08-26 | 北京邮电大学 | File transmission method based on third party message-oriented middle-ware |
| CN106910146A (en) * | 2017-02-28 | 2017-06-30 | 东北师范大学 | A kind of isomery educational data switching plane and method based on Stream Processing technology |
| CN107438061A (en) * | 2016-05-27 | 2017-12-05 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of kafka clients authentication |
-
2018
- 2018-07-26 CN CN201810833542.5A patent/CN108833443B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080083026A1 (en) * | 2006-10-02 | 2008-04-03 | Bea Systems, Inc. | Kerberos Protocol Security Provider for a Java Based Application Server |
| CN104869174A (en) * | 2015-06-15 | 2015-08-26 | 北京邮电大学 | File transmission method based on third party message-oriented middle-ware |
| CN107438061A (en) * | 2016-05-27 | 2017-12-05 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of kafka clients authentication |
| CN106910146A (en) * | 2017-02-28 | 2017-06-30 | 东北师范大学 | A kind of isomery educational data switching plane and method based on Stream Processing technology |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110806958A (en) * | 2019-10-24 | 2020-02-18 | 长城计算机软件与系统有限公司 | Monitoring method, monitoring device, storage medium and electronic equipment |
| CN110995851A (en) * | 2019-12-11 | 2020-04-10 | 贝壳技术有限公司 | Message processing method, device, storage medium and equipment |
| CN110995851B (en) * | 2019-12-11 | 2021-12-24 | 贝壳找房(北京)科技有限公司 | Message processing method, device, storage medium and equipment |
| CN111262915A (en) * | 2020-01-10 | 2020-06-09 | 北京东方金信科技有限公司 | Kafka cluster-crossing data conversion system and method |
| CN111262915B (en) * | 2020-01-10 | 2020-09-22 | 北京东方金信科技有限公司 | Kafka cluster-crossing data conversion system and method |
| CN111866189A (en) * | 2020-09-03 | 2020-10-30 | 中国银行股份有限公司 | Batch file transmission method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108833443B (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108833443A (en) | A kind of method for message transmission and system, computer equipment | |
| CN104980450B (en) | A kind of message delivery method and system and message processing device | |
| CN106953718A (en) | A kind of implementation method, device and the equipment of the retransmission process of code block | |
| CN104348697B (en) | The processing method of video communication information, apparatus and system | |
| CN104917586A (en) | Verification method, device and system of transmitted data | |
| CN104348578B (en) | The method and device of data processing | |
| CN103259797A (en) | Data file transmission method and platform | |
| CN114338793A (en) | Message pushing method and device, electronic equipment and readable storage medium | |
| CN103428260A (en) | System and method for allocating server to terminal and efficiently delivering messages to the terminal | |
| CN108040266A (en) | Abnormality eliminating method, device and the storage medium of data synchronization | |
| CN103067184B (en) | The abnormality eliminating method of offline charging and system | |
| CN107294704B (en) | Password generation method, password execution method and terminal | |
| JP5961471B2 (en) | Output comparison method in multiple information systems | |
| CN106302432B (en) | A kind of communication device and control method based on car networking | |
| CN110995780A (en) | API calling method and device, storage medium and electronic equipment | |
| CN108289165B (en) | Method and device for realizing camera control based on mobile phone and terminal equipment | |
| CN109743746A (en) | A kind of two-way converting detection BFD parameter consultation method, device and chip | |
| CN106559470A (en) | A kind of method for pushing and device of account information | |
| CN113014610B (en) | Remote access method, device and system | |
| CN103490874A (en) | Safety information interaction system, method, device and server | |
| CN112732660A (en) | Intervention type file transmission method, device and system | |
| CN105187511A (en) | Method and device for logging applications on multiple terminals | |
| CN101741629A (en) | Communication encryption method supporting remote monitoring system | |
| CN106648444A (en) | Message processing method and device | |
| CN113259108A (en) | Certificate updating method, Internet of things platform and Internet of things equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100190 17-19 / F, building a 1, 66 Zhongguancun East Road, Haidian District, Beijing Patentee after: New Great Wall Technology Co.,Ltd. Address before: 100190 17-19 / F, building a 1, 66 Zhongguancun East Road, Haidian District, Beijing Patentee before: GREAT WALL COMPUTER SOFTWARE & SYSTEMS Inc. |
|
| CP01 | Change in the name or title of a patent holder |