CN111949417B - Message transmission method and storage medium - Google Patents

Message transmission method and storage medium Download PDF

Info

Publication number
CN111949417B
CN111949417B CN202010637102.XA CN202010637102A CN111949417B CN 111949417 B CN111949417 B CN 111949417B CN 202010637102 A CN202010637102 A CN 202010637102A CN 111949417 B CN111949417 B CN 111949417B
Authority
CN
China
Prior art keywords
water level
message
client
server
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010637102.XA
Other languages
Chinese (zh)
Other versions
CN111949417A (en
Inventor
刘德建
林伟
郭玉湖
陈宏�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujian Tianquan Educational Technology Ltd
Original Assignee
Fujian Tianquan Educational Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujian Tianquan Educational Technology Ltd filed Critical Fujian Tianquan Educational Technology Ltd
Priority to CN202010637102.XA priority Critical patent/CN111949417B/en
Publication of CN111949417A publication Critical patent/CN111949417A/en
Application granted granted Critical
Publication of CN111949417B publication Critical patent/CN111949417B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/547Messaging middleware
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/548Queue

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention provides a message sending method and a storage medium, wherein the method comprises the following steps: the client periodically acquires water level information of the MQ server; when the acquired water level information exceeds a preset next highest water level, the client adopts a delay sending strategy to send a message; when the acquired water level information exceeds a preset high water level, the client writes the information to be sent into a local queue; more than two messages are obtained from the local queue, assembled into a message and then sent to the MQ server. The invention can relieve the pressure of the server, avoid blocking the server, effectively avoid the influence of the business of the client, and improve the user experience; meanwhile, the secondary high water level is sent in a multi-gear delay time mode, and compared with single delay time, the message response speed can be improved; furthermore, the adopted water level information acquisition mode can reduce the change to the original flow to the greatest extent, so that the error probability can be avoided, and the cost and the resource can be saved.

Description

Message transmission method and storage medium
Technical Field
The invention relates to the field of message middleware, in particular to a message sending method and a storage medium.
Background
Message middleware is suitable for distributed environments where reliable data transfer is required. And a message middleware mechanism system is adopted, and different objects activate events of the other party by transmitting messages, so that corresponding operations are completed. The sender sends the message to the message server, which stores the message in several queues and forwards the message to the receiver when appropriate. Message middleware can communicate between different platforms, is often used to mask features between various platforms and protocols, and enables collaboration between applications, which has the advantage of providing both synchronous and asynchronous connections between clients and servers, and can transfer or store forward messages at any time, which is why it is a further factor than remote procedure calls. In a general-purpose system, kafka or rubbitmq or other types of message middleware are typically used. Typically each message middleware functions substantially similarly, with a message sender sending messages to queues or topics from which specialized consumer threads obtain messages (topics in kafka, queues in rabitmq). Legacy systems typically use MQs in that messages are sent to queues by specialized message producers and are consumed asynchronously by backend consumers.
In the present message middleware, there is generally a concept of high and low water levels. Let us take the rubbi as an example, and simply say that the high and low water levels refer to the occupancy rate of the memory or disk in the rubbi. When the memory usage exceeds the configured threshold or the disk remaining space is lower than the configured threshold, the rubbitmq blocks the client connection and increases the message sending delay time, so as to avoid service breakdown. A scenario that is often encountered, i.e. when the rubbmq is already in a high water level state, the client continues to send a message to the rubbmq, which will cause blocking due to the high water level state, resulting in delay in sending the message, and thus affecting the call of the service.
In the prior art, as the application number is 201210490825.7, a method and a device for managing a queue are disclosed, wherein the method and the device monitor the idle rate or the filling rate of the queue, and store the data required to be added into the queue into a queue storage file in a disk when the idle rate or the filling rate exceeds a set first threshold; and when the filling rate of the queue is lower than a set second threshold value, loading the data in the queue storage file into the queue. By controlling the filling rate of the queue, simultaneous reading and writing can be prevented, and the continuous reading and writing performance of the magnetic disk can be better utilized due to batch writing and reading operation, so that the system load is reduced. However, the solution needs to have an independent monitoring program to monitor the idle rate and the filling rate, and all coarse grains are processed by the MQ server for the message, so that the problem of client blockage caused by message writing under high water level pressure of the server still cannot be solved. Because the IO of the message writing file is more time-consuming than the message writing into the memory queue, the blocking situation of the client is more serious.
Therefore, it is necessary to provide a solution to the problem of traffic congestion at the client caused by high water pressure at the server.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the message sending method and the storage medium are provided, the influence of the high water level and the low water level on the client service is effectively relieved, and the user experience is improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
a message sending method, comprising:
the client periodically acquires water level information of the MQ server;
when the acquired water level information exceeds a preset next highest water level, the client adopts a delay sending strategy to send a message;
when the acquired water level information exceeds a preset high water level, the client writes the information to be sent into a local queue; more than two messages are obtained from the local queue, assembled into a message and then sent to the MQ server.
The other technical scheme provided by the invention is as follows:
a computer readable storage medium having stored thereon a computer program which, when executed by a processor of a client, is capable of carrying out the steps comprised in a message sending method as described above.
The invention has the beneficial effects that: the water level information of the MQ server is obtained regularly, so that the mode of sending the information by the client is adjusted, and particularly, when the water level is high, the pressure of the server is relieved in a mode of sending the information in batches for a small number of times, the blocking of the server is avoided, the influence on the business of the client is effectively avoided, and therefore the user experience is improved.
Drawings
FIG. 1 is a flow chart of a message sending method according to an embodiment of the invention;
fig. 2 is a flow chart of a message sending method according to a first embodiment of the present invention.
Detailed Description
In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.
The most critical concept of the invention is as follows: according to the water level information of the MQ server, which is obtained regularly, the message sending mode of the client is dynamically adjusted, and particularly, when the water level is high, the pressure of the server is relieved in a mode of batch and few sending.
Referring to fig. 1, the present invention provides a message sending method, which includes:
the client periodically acquires water level information of the MQ server;
when the acquired water level information exceeds a preset next highest water level, the client adopts a delay sending strategy to send a message;
when the acquired water level information exceeds a preset high water level, the client writes the information to be sent into a local queue; more than two messages are obtained from the local queue, assembled into a message and then sent to the MQ server.
From the above description, the beneficial effects of the invention are as follows: the information of the high water level and the low water level of the server is obtained regularly, so that the rate of client message transmission is controlled according to a certain rule, and the influence on the service caused by the state of the high water level and the low water level can be relieved to a certain extent, and the experience is improved.
Further, the secondary high water level is a water level range and is divided into a first gradient range, a second gradient range and a third gradient range from low to high in sequence; and setting corresponding first delay time length, second delay time length and third delay time length in the delay transmission strategy corresponding to the first gradient range, the second gradient range and the third gradient range respectively.
From the above description, when the service end does not reach the high water level but the water level is not low, a gradient speed limiting mode is adopted to send a message, so that the pressure of the service end in the next-highest water level state is relieved, and the service end is prevented from entering the high water level state as much as possible.
Further, the client periodically obtains water level information of the MQ server, including:
the client periodically attaches a water level information acquisition mark to the message sent to the MQ server;
the client receives response information of the MQ server, wherein the response information corresponding to the information attached with the mark also comprises current water level information of the MQ server.
The above description shows that the reply of the server can be received only by attaching the specific mark together when the message is sent daily, and the water level information acquisition mode can reduce the modification to the original flow to the greatest extent, thereby avoiding the probability of error and saving cost and resources.
Further, the client writes the message to be sent into a local queue, and then further includes:
and if the local queue is full, discarding the message to be written, generating a log corresponding to the message to be written, and sending out alarm information.
As can be seen from the above description, when the local queue is also full, that is, the client and the server are in a relatively tight state, the server is ensured not to be blocked and the service is abnormal by discarding the message and leaving the warning; but also is favorable for the recovery of the bottom-left restarting.
Further, the method further comprises the following steps:
and when the acquired water level information does not exceed the preset next highest water level, the client sends a message to the MQ server.
From the above description, when the server side is in good state, the service is executed according to the original logic, so as to maintain the optimal service processing performance.
The other technical scheme provided by the invention is as follows:
a computer readable storage medium having stored thereon a computer program which, when executed by a processor of a client, is capable of performing the steps comprised in a message sending method comprising:
the client periodically acquires water level information of the MQ server;
when the acquired water level information exceeds a preset next highest water level, the client adopts a delay sending strategy to send a message;
when the acquired water level information exceeds a preset high water level, the client writes the information to be sent into a local queue; more than two messages are obtained from the local queue, assembled into a message and then sent to the MQ server.
Further, the secondary high water level is a water level range and is divided into a first gradient range, a second gradient range and a third gradient range from low to high in sequence; and setting corresponding first delay time length, second delay time length and third delay time length in the delay transmission strategy corresponding to the first gradient range, the second gradient range and the third gradient range respectively.
Further, the client periodically obtains water level information of the MQ server, including:
the client periodically attaches a water level information acquisition mark to the message sent to the MQ server;
the client receives response information of the MQ server, wherein the response information corresponding to the information attached with the mark also comprises current water level information of the MQ server.
Further, the client writes the message to be sent into a local queue, and then further includes:
and if the local queue is full, discarding the message to be written, generating a log corresponding to the message to be written, and sending out alarm information.
Further, the method further comprises the following steps:
and when the acquired water level information does not exceed the preset next highest water level, the client sends a message to the MQ server.
From the foregoing description, it will be appreciated by those skilled in the art that the foregoing embodiments may be implemented, in whole or in part, by hardware, by a computer program, where the program may be stored on a computer readable storage medium, where the program, when executed, may include the steps of the methods described above. After the program is executed by the processor, the beneficial effects corresponding to the methods can be realized.
The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
Example 1
Referring to fig. 2, the present embodiment provides an optimization method for message transmission, which is suitable for the interaction between the client and the MQ server, and can effectively alleviate the influence of the high-low water level state of the server on the service, so as to improve the user experience.
In this embodiment, a rubbi server is taken as an example to describe that when the rubbi server is in a high water level state of the server, the message sending is blocked, and the message sending delay is increased. Specifically, the method of the present embodiment includes:
s1: the client presets a secondary high water level and a high water level and message sending strategies corresponding to the secondary high water level and the high water level respectively.
In a specific example, the second highest water level corresponds to a water level range, for example, 60% -75%, or other ranges, and the specific range is set according to actual requirements; and different water level gradients are sequentially divided from low to high for the next-highest water level range, and corresponding different delay times are configured. For example, the next highest water level range is sequentially divided into a first gradient range, a second gradient range, and a third gradient range. For example, 60% -65% of the bits are in a first gradient range, 65% -70% are in a second gradient range; 70% -75% is the third gradient range. The corresponding message sending strategy is a delayed sending strategy, and corresponding first delay time length, second delay time length and third delay time length are respectively set in the delayed sending strategy corresponding to the first gradient range, the second gradient range and the third gradient range. The high water level corresponds to a specific value, e.g., 75%, which is determined based on historical empirical data for greater accuracy.
S2: the client periodically acquires water level information of the MQ server.
In a specific example, modification of the manner in which the client sends the message is specifically required. The old way is to send the message directly to the server and get the corresponding response information, only one field is successful or not in the response information. In this embodiment, on the basis of the old transmission mode, a getInfo flag, that is, a water level information acquisition flag, is periodically added to the message sent by the client; after the server receives the message containing the mark, the water level information at the moment of the server is returned to the response information at the same time. Wherein the periodic configuration supports flexible coordination. It can be seen that in this example, the client does not need to carry a flag every time a message is sent, but carries a flag when the corresponding time is reached according to the configured time, so that the water level information at the moment of the server can be obtained in a minimum modification manner.
S3: after the client acquires the water level information returned by the server, judging;
s4: if the acquired water level information does not exceed the preset next highest water level, namely still at the low water level, the client directly sends the information to the MQ server when sending the information later, namely, the old mode is maintained.
S5: if the acquired water level information exceeds the preset next highest water level, the client side sends a message by adopting a delay sending strategy;
specifically, it will be further judged which gradient range in the next highest water level the acquired water level information is located in:
if the acquired water level information corresponds to the first gradient range, the client delays sending the message after a first time when the message is put in each time;
when the acquired water level information corresponds to the second gradient range, the client delays sending the message for a second time when the message is put down each time;
when the acquired water level information corresponds to the third gradient range, the client delays sending the message for a third time every time when the message is put.
S6: if the acquired water level information exceeds a preset high water level, the client side does not directly call a server side interface to send the information, and writes the information to be sent into a local queue; then, the special asynchronous thread obtains more than two messages from the local queue, assembles the messages into a message and then sends the message to the MQ server; when the local queue is full and the client message can not be stored continuously, the message to be sent is recorded to the relevant log, the message is abandoned, and meanwhile, alarm information is sent.
In this step, the purpose of adopting the mode of writing into the local queue and sending the message through the asynchronous thread is to adopt the asynchronous queue to decouple the client from the server when the client is at the high water level, so as to prevent the blocking or delay of the client interface caused by the high water level state.
According to the method and the device for controlling the service level information, the sending rate of the client message can be dynamically adjusted according to the real-time water level information of the service side, the influence on the service caused by the high and low water level states of the service side can be effectively relieved to a certain extent, and therefore user experience is improved.
Example two
The present embodiment provides a specific application scenario corresponding to the first embodiment:
1. the client side carries a getInfo mark at intervals of 15 seconds when sending a message to the rubbitmq server side; when the server receives the field, the water level information of the server is returned.
2. Setting a high water level scene when the water level in the rubbitmq exceeds 75% at a client, and setting a next high water level scene when the water level proportion is between 60% and 75%;
3. after the client receives the server water level less than 60%, the client directly sends a message to the rubbitmq server by using the old mode.
4. When the water level is 60% -75%, namely the next highest water level scene is reached, the client sends the message by adopting a delay sending strategy. The delay time can be set into a plurality of gears, and different delay times are set according to different water level ratios of the next highest water level range. The specific delayed transmission strategy is assumed as follows: (the specific requirement is to prevent the excessive delay of the service interface caused by the delay according to the service interface usage scenario)
4.1 when the water level is 60% -65%, the client will delay 200 ms after each message transmission, and then send the message.
4.2 when the water level is 65% -70%, the client will delay the message transmission 400 ms after each time the message is transmitted.
When the water level is 70% -75%, the client transmits the message after each time the message is transmitted for 600 milliseconds.
5. When the water level exceeds 75%, namely, a high water level scene is reached, the client does not directly call a rubbitmq server interface to send a message, the message is modified to be directly written into a local queue, and a special asynchronous thread performs batch assembly of a plurality of messages for the message in the local queue and sends the message. At this time, the local asynchronous thread queries a plurality of messages (10 messages are assumed) in the local queue at a time, and after batch assembly of the 10 messages, sends a message to the rubbidmq server. The aim of the method is to adopt an asynchronous queue to decouple the client from the server when the client is at a high water level, prevent the blocking or delay of the client interface caused by the high water level state, and simultaneously reduce the number of times of message transmission and reduce the pressure of the rubbidq server. At this time, if the number of client messages exceeds the local queue, the relevant log is recorded, the messages are abandoned, and alarm information is sent out.
Example III
The present embodiment corresponds to the first and second embodiments, and provides a computer readable storage medium having a computer program stored thereon, where the computer program, when executed by a processor of a client, is capable of implementing the steps included in a message sending method described in the first or second embodiment. The specific steps will not be repeated here, and the details will be described in the first or second embodiment.
In summary, the message sending method and the storage medium provided by the invention can relieve the pressure of the server, avoid blocking the server, effectively avoid the influence on the service of the client, and further improve the user experience; meanwhile, the secondary high water level is sent in a multi-gear delay time mode, and compared with single delay time, the message response speed can be improved; furthermore, the adopted water level information acquisition mode can reduce the change to the original flow to the greatest extent, so that the error probability can be avoided, and the cost and the resource can be saved; further, the message exceeding the local queue in the high water level scene can carry out the bottom-left alarm, which is more beneficial to restarting recovery.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.

Claims (4)

1. A method of transmitting a message, comprising:
the client periodically acquires water level information of the MQ server;
when the acquired water level information exceeds a preset next highest water level, the client adopts a delay sending strategy to send a message;
when the acquired water level information exceeds a preset high water level, the client writes the information to be sent into a local queue; more than two messages are obtained from the local queue, assembled into a message and then sent to the MQ server;
the client writes the message to be sent into a local queue, and then further comprises:
if the local queue is full, discarding the message to be written, generating a log corresponding to the message to be written, and sending out alarm information;
the secondary high water level is a water level range and is divided into a first gradient range, a second gradient range and a third gradient range from low to high in sequence; and setting corresponding first delay time length, second delay time length and third delay time length in the delay transmission strategy corresponding to the first gradient range, the second gradient range and the third gradient range respectively.
2. The method for sending a message as claimed in claim 1, wherein the client periodically obtains water level information of the MQ server, comprising:
the client periodically attaches a water level information acquisition mark to the message sent to the MQ server;
the client receives response information of the MQ server, wherein the response information corresponding to the information attached with the mark also comprises current water level information of the MQ server.
3. The message transmission method as claimed in claim 1, further comprising:
and when the acquired water level information does not exceed the preset next highest water level, the client sends a message to the MQ server.
4. A computer readable storage medium having stored thereon a computer program, characterized in that the program, when being executed by a processor of a client, is capable of realizing the steps comprised in a message sending method according to any of the preceding claims 1-3.
CN202010637102.XA 2020-07-03 2020-07-03 Message transmission method and storage medium Active CN111949417B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010637102.XA CN111949417B (en) 2020-07-03 2020-07-03 Message transmission method and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010637102.XA CN111949417B (en) 2020-07-03 2020-07-03 Message transmission method and storage medium

Publications (2)

Publication Number Publication Date
CN111949417A CN111949417A (en) 2020-11-17
CN111949417B true CN111949417B (en) 2024-03-01

Family

ID=73339897

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010637102.XA Active CN111949417B (en) 2020-07-03 2020-07-03 Message transmission method and storage medium

Country Status (1)

Country Link
CN (1) CN111949417B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112988417A (en) * 2021-03-04 2021-06-18 长沙市到家悠享网络科技有限公司 Message processing method and device, electronic equipment and computer readable medium
CN113783939A (en) * 2021-08-20 2021-12-10 奇安信科技集团股份有限公司 File transmission method and device, electronic equipment and storage medium
CN115643309A (en) * 2022-10-27 2023-01-24 极道科技(北京)有限公司 Dynamic flow control method and system based on request scheduling queue length

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105004396A (en) * 2015-05-06 2015-10-28 湖南大麓管道工程有限公司 Water level monitoring method
CN107391271A (en) * 2017-05-17 2017-11-24 阿里巴巴集团控股有限公司 A kind of delayed tasks triggering method and device based on Message Queuing system
CN108388479A (en) * 2018-02-10 2018-08-10 深圳壹账通智能科技有限公司 Postpone information push method, device, computer equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8516054B2 (en) * 2000-12-20 2013-08-20 Aurea Software, Inc. Message handling

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105004396A (en) * 2015-05-06 2015-10-28 湖南大麓管道工程有限公司 Water level monitoring method
CN107391271A (en) * 2017-05-17 2017-11-24 阿里巴巴集团控股有限公司 A kind of delayed tasks triggering method and device based on Message Queuing system
CN108388479A (en) * 2018-02-10 2018-08-10 深圳壹账通智能科技有限公司 Postpone information push method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN111949417A (en) 2020-11-17

Similar Documents

Publication Publication Date Title
CN111949417B (en) Message transmission method and storage medium
CN107391271B (en) Message queue system-based delayed task triggering method and device
CN112507029B (en) Data processing system and data real-time processing method
US20120016994A1 (en) Distributed system
US10243855B2 (en) Efficient reliable distributed flow-controlled event propagation
CN107872396B (en) Communication method with timeout control, processing device, terminal device and server
CN111078426A (en) High concurrency solution under back-end micro-service architecture
CN109240836B (en) Method and device for configuring messages of message queue
CN103634224A (en) Method and system for transmitting data in network
CN113703954A (en) Message backup method and device, electronic equipment and computer storage medium
CN115211092B (en) Message pulling method and device and computer storage medium
CN111045837B (en) Cross-service consumption method and storage medium
CN113467969B (en) Method for processing message accumulation
CN111526188A (en) System and method for ensuring zero data loss based on Spark Streaming in combination with Kafka
CN109756464B (en) Communication method, server and client
US20020028679A1 (en) Data transmission based on available wireless bandwidth
US20050278166A1 (en) Data distribution apparatus, its control method, program, and storage medium
US11212220B2 (en) Optimized reconciliation in a controller-switch network
CN114598662A (en) Message queue cluster federal management system and method
CN115499501A (en) Message pushing method, system, service gateway and storage medium
CN110572474B (en) Method for embedded terminal long-connection communication
CN113918364A (en) Redis-based lightweight message queue processing method and device
US7844578B2 (en) Parallel archiving for a messaging system
CN114979285B (en) Service calling method, device, equipment, system, storage medium and product
CN113141319B (en) Message processing method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant