CN111475759A - Message pushing platform, method, device, server and storage medium - Google Patents

Message pushing platform, method, device, server and storage medium Download PDF

Info

Publication number
CN111475759A
CN111475759A CN202010268157.8A CN202010268157A CN111475759A CN 111475759 A CN111475759 A CN 111475759A CN 202010268157 A CN202010268157 A CN 202010268157A CN 111475759 A CN111475759 A CN 111475759A
Authority
CN
China
Prior art keywords
message
push
pushing
pushed
messages
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010268157.8A
Other languages
Chinese (zh)
Inventor
冷建正
李远水
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Baiguoyuan Information Technology Co Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to CN202010268157.8A priority Critical patent/CN111475759A/en
Publication of CN111475759A publication Critical patent/CN111475759A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/958Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention discloses a message pushing platform, a message pushing method, a message pushing device, a message pushing server and a storage medium. Wherein, this platform includes: the system comprises a message publishing and subscribing component, a message consuming process and a message pushing process, wherein the message publishing and subscribing component stores pushing messages under different service applications; the message consumption process pulls the push message stored in the message publishing and subscribing component in parallel; and the message pushing process generates pushing contents corresponding to all the pushing messages according to the pushing characteristics under the service application related to the pushing messages pulled by the message consumption process. According to the technical scheme provided by the embodiment of the invention, the message publishing and subscribing components are used for storing the push messages under different business applications, so that the push omission caused by the loss of the push messages due to the restarting or downtime of the service is avoided, a business processing thread is not required to be adopted for sequentially processing all the push messages, the time consumed by the waiting of the push messages is avoided, and the message push instantaneity is improved.

Description

Message pushing platform, method, device, server and storage medium
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a message pushing platform, a message pushing method, a message pushing device, a server and a storage medium.
Background
With the rapid development of internet technology, a phenomenon of pushing a corresponding service notification message to a user occurs in various service scenes, for example, after a main broadcast starts a live broadcast on a live broadcast application program, a live broadcast background generates a corresponding live broadcast start notification and pushes the live broadcast start notification to fans of the main broadcast, so that the fans of the main broadcast can participate in the live broadcast as soon as possible.
After generating a corresponding push message, an existing message push background generally caches the push message in an assigned process, and then sequentially processes each cached push message by starting a batch of service processing threads. At this time, because each push message is processed in only one service processing thread, and when the user list to be pushed is too large, the service processing thread is required to traverse the user list, so that the push message is pushed to each user in the user list, the time consumed for pushing is too long, other push messages after the push message are blocked from being processed in sequence, and the requirement of message pushing on real-time property cannot be met; meanwhile, in the process of processing the push messages, a situation that the cache thread is restarted or the background system is down occurs, and a large amount of unprocessed push messages are lost, so that the problem of push omission is caused.
Disclosure of Invention
The embodiment of the invention provides a message pushing platform, a message pushing method, a message pushing device, a server and a storage medium, which can avoid pushing omission caused by loss of a pushed message and improve the instantaneity of message pushing.
In a first aspect, an embodiment of the present invention provides a message pushing platform, where the message pushing platform includes: the system comprises a message publishing and subscribing component, a message consuming process and a message pushing process, wherein the message publishing and subscribing component stores pushing messages under different service applications; wherein the content of the first and second substances,
the message consumption process pulls the push messages stored in the message publishing and subscribing component in parallel; and the message pushing process generates pushing contents corresponding to all the pushing messages according to pushing characteristics under the service application related to the pushing messages pulled by the message consumption process.
In a second aspect, an embodiment of the present invention provides a message pushing method, which is applied to the message pushing platform provided in the first aspect, and the method includes:
the method comprises the steps that push messages stored by a Kafka message system facing different service applications are pulled in parallel through a message consumption process;
and concurrently generating push contents corresponding to each push message according to the push characteristics under the service application associated with each push message through the message push process.
In a third aspect, an embodiment of the present invention provides a message pushing apparatus, which is disposed in the message pushing platform provided in the first aspect, and includes:
the message pulling module is used for pulling the push messages stored by the Kafka message system facing different service applications in parallel through a message consumption process;
and the push content generation module is used for generating push contents corresponding to all the push messages through the message push process according to the push characteristics under the service application associated with each push message.
In a fourth aspect, an embodiment of the present invention provides a server, where the server includes:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the message pushing method according to any embodiment of the present invention.
In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a message pushing method according to any embodiment of the present invention.
According to the message pushing platform, the method, the device, the server and the storage medium provided by the embodiment of the invention, the message pushing under different service applications is stored through the message publishing and subscribing component, so that the message consuming process can pull and the message pushing process can execute corresponding pushing operation, pushing omission caused by loss of the message pushing due to service restart or downtime is avoided, meanwhile, the message consuming process pulls the message pushing stored in the message publishing and subscribing component in parallel, and then the message pushing process generates the pushing content corresponding to each pushing message according to the pushing characteristics under the service application related to each pulled message, and does not need to adopt a service processing thread to process each pushing message in sequence, so that the time consumption of message waiting is avoided, and the message pushing instantaneity is improved.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
fig. 1 is a schematic architecture diagram of a message push platform according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a message push platform according to a second embodiment of the present invention;
fig. 3 is a flowchart of a message pushing method according to a third embodiment of the present invention;
fig. 4 is a flowchart of a message pushing method according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a message pushing apparatus according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a server according to a sixth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict.
Example one
Fig. 1 is a schematic architecture diagram of a message push platform according to an embodiment of the present invention. The embodiment can be applied to any situation that the message pushing requirement exists under different business applications. Specifically, referring to fig. 1, the message push platform may include: a message publish subscribe component 110, a message consume process 120, and a message push process 130.
Wherein, the message publishing and subscribing component 110 stores push messages under different service applications; the message consumption process 120 pulls the push messages stored in the message publish-subscribe component 110 in parallel; the message pushing process 130 generates pushing content corresponding to each pushing message according to the pushing characteristics of the pushing message related to the service application pulled by the message consuming process 120.
Optionally, the message publishing and subscribing component 110 in this embodiment may be a Kafka message system, which is a high-throughput distributed publishing and subscribing message system, and can support high-speed reading and writing of push messages under different service applications, and ensure a processing rate of the push messages.
In this embodiment, a service Application refers to an Application program (App) that executes a corresponding service function in different service scenarios, such as a live App or an information push App, and at this time, when a user uploads a dynamic state in a corresponding service Application, a background server needs to notify the dynamic state to a relevant user of the user, so that various push messages may exist in different service applications, in this embodiment, each push message in different service applications is stored by the message publishing and subscribing component 110, the subsequent message consuming process 120 may pull the corresponding push message in parallel in the message publishing and subscribing component 110, determine, by the message pushing process 130, a push feature under the service Application associated with each push message pulled by the message consuming process 120, and further send each push message, generate a push content corresponding to the push message according to the push feature corresponding to the push message, the real-time property of message pushing is improved through the concurrent message pushing consumption.
Meanwhile, in the message pushing process, the server inevitably has a restart or downtime condition, and at this time, the message publishing and subscribing component 110 (such as a Kafka message system) is adopted in the embodiment to store the push messages under different service applications, so that even if the server restarts or crashes, the push messages stored on the server are not lost, thereby avoiding push omission caused by push message loss.
Specifically, for different service applications and message types to which each push message belongs, the message publishing and subscribing component 110 stores each push message in different storage regions, for example, a corresponding message type (topic) is preset in the Kafka message system, and further partitions (partition) the published message under each message type (topic), and multiple partitions exist under one message type; at this time, the Kafka message system may correspondingly store each push message in each partition under different message categories, that is, different push messages are directed to corresponding storage locations in the message publishing and subscribing component 110.
At this time, in order to ensure the efficiency of the message consuming process 120 pulling each push message from the message publishing and subscribing component 110 in parallel, in this embodiment, the message consuming process 120 employs a message thread pool, and the message thread pool may include two or more message pulling threads 121, which are used to pull push messages in different storage locations from the message publishing and subscribing component 110 in parallel. That is to say, the plurality of message pull threads 121 under the message consumption process 120 may concurrently perform corresponding message pull operations, and each message pull thread 121 may concurrently pull the push messages in different storage locations in the message publish-subscribe component 110, thereby ensuring the pull efficiency of the push messages.
In addition, after acquiring the push message pulled by the message consumption process 120, the message push process 130 may generate push content corresponding to each push message according to the push characteristics under the service application associated with each push message, at this time, the message push process 130 may include two or more message processing threads 131, and the service application associated with each push message pulled by the message consumption process 120 is analyzed synchronously and concurrently by the message processing threads 131, so that the efficiency of push message consumption is ensured, and the wait time of push message consumption is avoided; after determining a service application associated with a push message, a certain message processing thread 131 may invoke a service processing process (e.g., app _ push _ handle) on the service application associated with the push message to determine a push feature of the service application, and further generate push content corresponding to the push message by using the push feature; at this time, the push characteristics of the service application may be a push format requirement unique to the service application and a contained push document type or picture, and the push characteristics of different service applications may be maintained by a developer of the service application.
Further, after the message pushing process 130 concurrently generates the pushing content corresponding to each pushing message, the message pushing process 130 needs to send the pushing content to the corresponding pushed party to notify each pushed party of obtaining the relevant concerned message, so that in this embodiment, after determining the service application associated with each pushing message pulled by the message consuming process 120 each time, the message pushing process 130 is further configured to determine the pushed list facing each pushing message pulled by the message consuming process 120 under the corresponding service application, so as to respectively push the pushing content corresponding to each pushing message to each pushed party in the pushed list facing each pushing message.
Specifically, the message pushing process 130 obtains a pushed list facing each push message under the corresponding service application by analyzing each other user associated with the push party where each push message is located under the corresponding service application, where the pushed list includes each pushed party indicating that the push message needs to be delivered, and then by traversing the pushed list facing each push message, the push content corresponding to each push message can be pushed to each pushed party in the pushed list facing each push message.
Further, since the number of the pushed parties associated with each of the pushing parties of the pushing messages under different service applications is different, so that the lengths of the pushed lists facing different pushing messages are different, at this time, the time consumed for traversing the pushed lists facing each of the pushing messages is different, and traversal omission is easily caused when the time consumption is too long, in this embodiment, after determining the pushed list facing each of the pushing messages, it is first determined whether the entire length of the pushed list exceeds the preset length of the list, where the preset length of the list is the preset upper limit of the list capable of ensuring traversal integrity, and if the entire length of the pushed list facing a certain pushing message exceeds the preset length of the list, the detachable form of the pushed list is first determined according to the position searched by each of the pushed parties in the pushed list, for example, if there are 20 pushed parties in the pushed list and the preset list length is 15 pushed parties, at this time, the first 10 pushed parties in the pushed list are found from the redis system, and the last 10 pushed parties are found from other storage systems, at this time, the pushed list is splittable in such a manner that the first 10 pushed parties and the last 10 pushed parties are respectively split into two new pushed lists; therefore, in order to ensure the traversal integrity of the pushed list to which the push message faces, the push message may be correspondingly converted into push message copies of an appointed number, where the appointed number is consistent with the number of lists that are allowed to be split in the detachable form of the pushed list, so as to map each push message copy with each list split result obtained after the pushed list is split in the detachable form one by one, and simultaneously store each push message copy as a new push message into the message publishing and subscribing component 110 again, so as to perform pushing continuously; at this time, the pushed list to which each pushed message copy faces under the corresponding service application is a target list splitting result matched with each list splitting result of the pushed message copy after the pushed list is split according to a split form.
It should be noted that the list length of each list splitting result obtained after the pushed list is split according to the detachable form does not exceed the preset list length.
Meanwhile, in order to avoid repeated pushing of the push message in the message publishing and subscribing component 110, in this embodiment, after a push content corresponding to a certain push message is generated each time, the push message is marked in the message publishing and subscribing component 110 to indicate that the push message has been pushed, and does not need to be pulled again; at this time, the message pushing platform in this embodiment may further include: the message management process 140 and the message update table 150 corresponding to each preset storage location in the message publishing and subscribing component 110, after the message pushing process 130 generates the push content of the corresponding push message each time, the message publishing and subscribing component 110 records the storage identifier set for the push message in the message publishing and subscribing component 110 in the message update table 150 corresponding to the storage location of the push message in the message publishing and subscribing component 110; the message management process 140 periodically reads the storage identifier of the latest push message that is continuously consumed in each message update table, and submits the storage identifier to the message publishing and subscribing component 110.
For example, the storage identifier set by the message publishing subscription component 110 for each push message in the present embodiment can be the offset number of the push message; the message update table 150 corresponding to the storage location of the push message in the message publish-subscribe component 110 can be the corresponding offset ring.
Specifically, the message publishing and subscribing component 110 sets a corresponding storage identifier for each stored push message, and after each time the message pushing process 130 generates the push content corresponding to the push message, records the storage identifier of the push message into the message update table 150 corresponding to the storage location of the push message in the message publishing and subscribing component 110, so as to mark the push message of the storage identifier directly in the corresponding storage location of the message publishing and subscribing component 110 in the following; meanwhile, the message management process 140 reads the storage identifier of the latest push message that is continuously consumed in each message update table at regular time, and submits the storage identifier to the message publishing and subscribing component 110, where the storage identifier of the latest push message that is continuously consumed at this time can indicate that each push message located in the storage location in the message publishing and subscribing component 110 before the storage identifier of the latest push message has already completed consuming pushing, so that the message publishing and subscribing component 110 confirms that each push message located in the storage location before the storage identifier of the latest push message does not need to be pulled by the message consuming process 120, thereby avoiding repeated consumption of push messages.
According to the technical scheme provided by the embodiment, the push messages under different service applications are stored through the message publishing and subscribing component, so that the message consuming process pulls and the message pushing process executes corresponding pushing operation, pushing omission caused by loss of the push messages due to service restart or downtime is avoided, meanwhile, the message consuming process pulls the push messages stored in the message publishing and subscribing component in parallel, and then the message pushing process generates the push content corresponding to each push message according to the pushing characteristics under the service application associated with each pulled push message, so that each push message is not required to be sequentially processed by one service processing thread, the time consumed for waiting of the push messages is avoided, and the real-time performance of message pushing is improved.
Example two
Fig. 2 is a schematic structural diagram of a message push platform according to a second embodiment of the present invention. The embodiment is optimized on the basis of the technical scheme provided by the embodiment. Referring to fig. 2, the message push platform may include a message publishing subscription component 201, a message consuming process 202, a message pushing process 203, a message managing process 204, and a message update table 205 corresponding to each storage location preset in the message publishing subscription component 201.
The message publishing and subscribing component 201 stores push messages under different service applications, and the message consuming process 202 pulls the push messages stored in the message publishing and subscribing component 201 in parallel; the message pushing process 203 concurrently generates a pushing content corresponding to each pushing message according to the pushing feature under the service application associated with the pushing message pulled by the message consuming process 202. After generating the push content corresponding to the push message each time, the message push process 203 records the storage identifier set by the message publishing and subscribing component 201 for the push message into the message update table 205 corresponding to the storage location of the push message in the message publishing and subscribing component 201; the message management process 204 periodically reads the storage identifier of the latest push message continuously completing consumption in each message update table, and submits the storage identifier to the message publishing and subscribing component 201.
Meanwhile, the message push platform 20 in this embodiment may further include a message cache queue 206, a push front-end process 207, a message writing process 208, a content push component 209, and a content issuing process 210, in addition to the message publishing subscription component 201, the message consuming process 202, the message push process 203, the message management process 204, and the message update table 205 corresponding to each preset storage location in the message publishing subscription component 201.
Specifically, the message cache queue 206 is mainly used for caching push messages pulled by the message consumption process 202 from the message publishing and subscribing component 201 in parallel; at this time, the message pushing process 203 may include two or more message processing threads, which are used to concurrently read the push message cached in the message cache queue 206, and generate the push content corresponding to the push message according to the push characteristics under the service application associated with each push message. That is, after pulling out the push messages at different storage locations in parallel from the message publishing and subscribing component 201, the message consuming process 202 may sequentially cache the push messages in the message cache queue 206, and then each message processing thread in the message pushing process 203 may concurrently read each push message from the message cache queue 206, and generate the push content corresponding to each push message according to the push characteristics of the service application associated with each push message.
Optionally, when a certain user uploads a dynamic state in a corresponding service application, the background server notifies the relevant user of the dynamic state, and for a trigger operation of the user on different service applications, the push front-end process 207 receives push requests under different service applications, generates push messages corresponding to the push requests, and further invokes the message write-in process 208 to sequentially write each push message into different storage locations in the message publishing and subscribing component 201, so that the subsequent message consumption process 202 pulls the corresponding push message from the different storage locations in the message publishing and subscribing component 201 in parallel, and sends and generates push content corresponding to each push message according to push characteristics under the service application associated with each push message through the message push process 203.
Further, after generating the push content corresponding to each push message, the message push process 203 correspondingly stores each push content into the content push component 209, and then the content issue process 210 continuously reads the push content corresponding to each push message from the content push component 209, and issues each push content to each pushed party in the pushed list to which the push message corresponding to the push content faces under the corresponding service application, so as to implement comprehensive push of the push message.
According to the technical scheme provided by the embodiment, push messages under different service applications are stored through the message publishing and subscribing component, so that the message consuming process and the message pushing process execute corresponding pushing operations, pushing omission caused by loss of the push messages due to service restart or downtime is avoided, meanwhile, the message consuming process stores the push messages stored in the message publishing and subscribing component into the message cache queue through parallel pulling, reads all the push messages in the message cache queue through the message pushing process, determines the pushing characteristics under the service application associated with each push message, generates push contents corresponding to all the push messages and stores the push contents into the content pushing component, and subsequently, each push message is not required to be sequentially processed through one service processing thread, waiting time consumption of the push messages is avoided, and meanwhile, each push content under the content pushing component is respectively issued to the push message corresponding to the push content through the content issuing process and the push contents corresponding to the push message under the push contents are respectively issued to the corresponding service application And each pushed party in the pushed list facing under the service application ensures the comprehensiveness of message pushing and improves the instantaneity of message pushing.
EXAMPLE III
Fig. 3 is a flowchart of a message pushing method provided in a third embodiment of the present invention, and this embodiment is applicable to any situation where there is a message pushing requirement in different service applications, and is applied to the message pushing platform provided in the foregoing embodiment. The message pushing method provided by this embodiment may be executed by a message pushing apparatus provided by the embodiment of the present invention, and the apparatus may be implemented by software and/or hardware and integrated in a server executing the method.
Specifically, referring to fig. 3, the method may include the steps of:
s310, the Kafka message system is pulled in parallel through the message consumption process to push messages stored under different service applications.
Specifically, in this embodiment, a Kafka message system is used as a message publish-subscribe component to store push messages in different service applications, and the Kafka message system is a high-throughput distributed publish-subscribe message system, and can support high-speed reading and writing of push messages in different service applications, and ensure the processing rate of the push messages. At this time, in the message pushing process, the server inevitably has a restart or downtime condition, and at this time, the Kafka message system is adopted in the embodiment to store the push messages under different service applications, so that even if the server restarts or crashes, the push messages stored on the server are not lost, thereby avoiding push omission caused by push message loss.
For different service applications and message types to which each push message belongs, a corresponding message type (topic) is preset in a Kafka message system, and the issued message is further partitioned under each message type (topic), wherein a plurality of partitions exist under one message type; at this time, the Kafka message system may store each push message in each partition under different message categories, that is, different push messages may be directed to different storage locations in the Kafka message system.
Optionally, in this embodiment, the push message under different service applications may be pulled in parallel from the Kafka message system through the message consumption process. To ensure the efficiency of the message consuming process pulling individual push messages in parallel from the Kafka message system, the message consuming process may employ a message thread pool, which may include two or more message pull threads for pulling push messages in different memory locations in parallel from the Kafka message system. That is to say, a plurality of message pull threads in the message consumption process can concurrently execute corresponding message pull operations, and each message pull thread can concurrently pull push messages in different storage locations in the Kafka message system, thereby ensuring the pull efficiency of the push messages.
Further, before the pushing messages stored by the Kafka message system facing different service applications are pulled in parallel by the message consumption process in the embodiment, the method may further include: and generating corresponding push messages according to push requests under different service applications, and writing the push messages into a Kafka message system.
Specifically, the service application in this embodiment refers to an application program that executes a corresponding service function in different service scenarios, such as a live App or an information push App, and at this time, when a certain user uploads a piece of dynamic information in the corresponding service application, the background server needs to notify the piece of dynamic information to a relevant user of the user, so that, in this embodiment, for a trigger operation of the user on different service applications, a push request under different service applications is received through a push front-end process, and a push message corresponding to the push request is generated, and meanwhile, a message write process is invoked to write each push message in sequence to different storage locations in the Kafka message system, so that a subsequent message consumption process pulls the corresponding push message in parallel from different storage locations in the Kafka message system.
In addition, after the push messages under different service applications are pulled out from the Kafka message system in parallel through the message consumption process, the push messages can be sequentially cached in the message cache queue, so that the message push process can read all the push messages from the message cache queue concurrently.
And S320, generating push contents corresponding to each push message according to the push characteristics under the service application associated with each push message through the message push process.
Specifically, push messages which are parallelly pulled out from a Kafka message system by a message consumption process are continuously read from a message cache queue by the message pushing process, the pushing characteristics under the service application associated with the push messages are determined for each push message, and then the pushing content corresponding to each push message is generated according to the pushing characteristics corresponding to each push message, at the moment, the message pushing process can comprise two or more message processing threads, and the service application associated with each push message pulled by the message consumption process is synchronously and hairly analyzed by the message processing threads, so that the high efficiency of push message consumption is ensured, and the waiting time consumption of push message consumption is avoided; at this time, the push characteristics of the service application may be a push format requirement unique to the service application and a contained push document type or picture, and the push characteristics of different service applications may be maintained by a developer of the service application.
In addition, after concurrently generating push content corresponding to each push message through a message push process, the embodiment may further include: and respectively issuing each push content to each pushed party in a pushed list facing the push message corresponding to the push content under the corresponding service application.
Specifically, each other user associated with the push party where each push message is located under the corresponding service application is analyzed through the message push process to serve as the pushed party of the push message, so as to form a pushed list facing each push message under the corresponding service application, each pushed party indicating that the push message needs to be issued is included in the pushed list, and then, by traversing each pushed list facing each push message, the push content corresponding to each push message can be respectively pushed to each pushed party in the pushed list facing each push message.
According to the technical scheme provided by the embodiment, the push messages under different service applications are stored through the message publishing and subscribing component, so that the message consuming process pulls and the message pushing process executes corresponding pushing operation, pushing omission caused by loss of the push messages due to service restart or downtime is avoided, meanwhile, the message consuming process pulls the push messages stored in the message publishing and subscribing component in parallel, and then the message pushing process generates the push content corresponding to each push message according to the pushing characteristics under the service application associated with each pulled push message, so that each push message is not required to be sequentially processed by one service processing thread, the time consumed for waiting of the push messages is avoided, and the real-time performance of message pushing is improved.
Example four
Fig. 4 is a flowchart of a message pushing method according to a fourth embodiment of the present invention. The embodiment is optimized on the basis of the embodiment. Specifically, as shown in fig. 4, the present embodiment explains in detail other message processing steps existing in the small push process.
Optionally, as shown in fig. 4, this embodiment may specifically include the following steps:
s410, the Kafka message system is pulled in parallel through the message consumption process to push messages stored under different service applications.
S420, determining a pushed list of each push message facing under the corresponding service application through the message pushing process, so as to respectively push the push content corresponding to each push message to each pushed party in the pushed list facing to the push message.
Specifically, after the push content corresponding to each push message is concurrently generated through the message push process, the push content is also sent to the corresponding pushed party to inform each pushed party of acquiring the corresponding concerned message, therefore, after the message pushing process in this embodiment determines the service application associated with each pushed message pulled by the message consuming process each time, and also obtains a pushed list facing each pushed message under the corresponding service application by analyzing each other user related to the pushing party under the corresponding service application of each pushed message as the pushed party of the pushed message, the pushed list includes all the pushed parties indicating that the push message needs to be issued, and then the push message can be sent to the server by traversing the pushed list to which all the push messages are directed, the push content corresponding to each push message can be pushed to each pushed party in the pushed list to which the push message faces.
For example, because the number of pushed parties associated with each pushing party of the pushing message under different service applications is different, so that the lengths of pushed lists facing different pushing messages are different, at this time, the time consumed for traversing the pushed lists facing each pushing message is also different, and traversal omission is easily caused when the time consumption is too long, therefore, after determining the pushed list facing each pushing message, in this embodiment, after determining the pushed list facing each pushing message, it is also determined whether the whole length of the list of the pushed list exceeds the length of a preset list, where the length of the preset list is the preset upper limit of the list capable of ensuring traversal integrity; at this time, if the overall length of the list of the pushed list to which the push message faces under the corresponding service application exceeds the length of the preset list, the push message is converted into a specified number of push message copies according to the detachable form of the pushed list, the push message copies are stored into the Kafka message system as a new push message, and the corresponding message pushing operation is continuously executed on the new push message through the message consumption process and the message pushing process.
Specifically, if the overall length of a list of a pushed list to which a certain push message is directed exceeds the length of a preset list, determining a detachable form of the pushed list according to the positions searched by each pushed party in the pushed list, for example, if 20 pushed parties exist in the pushed list, the length of the preset list is 15 pushed parties, at this time, the first 10 pushed parties in the pushed list are found from a redis system, and the last 10 pushed parties are found from other storage systems, at this time, the detachable form of the pushed list is that the first 10 pushed parties and the last 10 pushed parties are respectively detached into two new pushed lists; therefore, in order to ensure the traversal integrity of the pushed list to which the push message faces, the push message can be correspondingly converted into push message copies with specified number, the specified number is consistent with the number of lists which are allowed to be split in the split form of the pushed list, so that each push message copy and each list splitting result obtained after the pushed list is split in the split form are mapped one by one, and meanwhile, each push message copy is stored into a Kafka message system again as a new push message for continuous pushing; at this time, the pushed list to which each pushed message copy faces under the corresponding service application is a target list splitting result matched with each list splitting result of the pushed message copy after the pushed list is split according to a split form.
It should be noted that the list length of each list splitting result obtained after the pushed list is split according to the detachable form does not exceed the preset list length.
S430, calling a service processing process under the service application associated with each push message through the message push process, and determining the push characteristics under the service application associated with each push message.
Specifically, in this embodiment, after determining a service application associated with a push message through a certain message processing thread in a message push process, a service processing process (e.g., app _ push _ handle) on the service application associated with the push message may be invoked to determine a push feature of the service application, and then generate push content corresponding to the push message by using the push feature; at this time, the push characteristics of the service application may be a push format requirement unique to the service application and a contained push document type or picture, and the push characteristics of different service applications may be maintained by a developer of the service application.
S440, according to the pushing characteristics under the business application associated with each pushing message, the pushing content corresponding to each pushing message is generated through the message pushing process.
S450, recording the storage identification set by the Kafka message system for the push message with the generated push content into a message update table corresponding to the storage position of the push message in the Kafka message system.
Optionally, to avoid repeated pulling and pushing of the push message in the Kafka message system, in this embodiment, after generating the push content corresponding to a certain push message each time, the push message is marked in the Kafka message system to indicate that the push message has been pushed, and the push message does not need to be pulled again; at this time, in order to distinguish the stored push messages, the Kafka message system sets a corresponding storage identifier for each push message, and after generating push content corresponding to the push message each time through a message push process, records the storage identifier of the push message in a message update table corresponding to a storage location of the push message in the Kafka message system, so as to mark the push message of the storage identifier directly in the corresponding storage location in the Kafka message system in the following.
And S460, regularly reading the storage identifier of the latest push message continuously completing consumption in each message update table through the message management process, and submitting the storage identifier to the Kafka message system.
Optionally, in this embodiment, after recording the storage identifier set in the push message that has generated the push content in the message update table corresponding to the storage location of the push message in the Kafka message system, the message management process may periodically read the storage identifier of the latest push message that has been continuously consumed in each message update table, and submit the storage identifier to the Kafka message system, where the storage identifier of the latest push message that has been continuously consumed at this time can indicate that each push message located before the storage identifier of the latest push message in the storage location in the Kafka message system has already completed consumption pushing, so that the Kafka message system confirms that each push message located before the storage identifier of the latest push message in the storage location does not need to be pulled by the message consumption process, thereby avoiding repeated consumption of push messages.
According to the technical scheme provided by the embodiment, the push messages under different service applications are stored through the message publishing and subscribing component, so that the message consuming process pulls and the message pushing process executes corresponding pushing operation, pushing omission caused by loss of the push messages due to service restart or downtime is avoided, meanwhile, the message consuming process pulls the push messages stored in the message publishing and subscribing component in parallel, and then the message pushing process generates the push content corresponding to each push message according to the pushing characteristics under the service application associated with each pulled push message, so that each push message is not required to be sequentially processed by one service processing thread, the time consumed for waiting of the push messages is avoided, and the real-time performance of message pushing is improved.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a message pushing apparatus according to a fifth embodiment of the present invention, which is disposed in the message pushing platform according to the fifth embodiment of the present invention, specifically, as shown in fig. 5, the apparatus may include:
the message pulling module 510 is configured to pull, in parallel, push messages stored by the Kafka message system for different service applications through a message consuming process;
the push content generating module 520 is configured to generate, through the message push process, push content corresponding to each push message according to the push characteristics of the service application associated with each push message.
According to the technical scheme provided by the embodiment, the push messages under different service applications are stored through the message publishing and subscribing component, so that the message consuming process pulls and the message pushing process executes corresponding pushing operation, pushing omission caused by loss of the push messages due to service restart or downtime is avoided, meanwhile, the message consuming process pulls the push messages stored in the message publishing and subscribing component in parallel, and then the message pushing process generates the push content corresponding to each push message according to the pushing characteristics under the service application associated with each pulled push message, so that each push message is not required to be sequentially processed by one service processing thread, the time consumed for waiting of the push messages is avoided, and the real-time performance of message pushing is improved.
The message pushing device provided by the embodiment can be applied to the message pushing method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
EXAMPLE six
Fig. 6 is a schematic structural diagram of a server according to a sixth embodiment of the present invention, as shown in fig. 6, the server includes a processor 60, a storage device 61, and a communication device 62; the number of the processors 60 in the server may be one or more, and one processor 60 is taken as an example in fig. 6; the processor 60, the storage device 61 and the communication device 62 in the server may be connected by a bus or other means, and the bus connection is taken as an example in fig. 6.
The storage device 61 is a computer-readable storage medium, and can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the message pushing method according to any embodiment of the present invention. The processor 60 executes various functional applications of the server and data processing by running software programs, instructions and modules stored in the storage device 61, that is, implements the message push method described above.
The storage device 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage device 61 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the storage device 61 may further include memory located remotely from the processor 50, which may be connected to a server over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication means 62 may be used to enable network connections or mobile data connections between devices.
The server provided by this embodiment can be used to execute the message pushing method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
EXAMPLE seven
Seventh, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the message pushing method in any of the above embodiments. The method specifically comprises the following steps:
the method comprises the steps that push messages stored by a Kafka message system facing different service applications are pulled in parallel through a message consumption process;
and concurrently generating push contents corresponding to each push message according to the push characteristics under the service application associated with each push message through the message push process.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the message pushing method provided by any embodiment of the present invention.
Based on the understanding that the technical solutions of the present invention can be embodied in the form of software products, such as floppy disks, Read-Only memories (ROMs), Random Access Memories (RAMs), flash memories (F L ASHs), hard disks or optical disks of a computer, etc., and include instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the message pushing apparatus, the units and modules included in the embodiment are only divided according to the functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (19)

1. A message push platform, comprising: the system comprises a message publishing and subscribing component, a message consuming process and a message pushing process, wherein the message publishing and subscribing component stores pushing messages under different service applications; wherein the content of the first and second substances,
the message consumption process pulls the push messages stored in the message publishing and subscribing component in parallel; and the message pushing process generates pushing contents corresponding to all the pushing messages according to pushing characteristics under the service application related to the pushing messages pulled by the message consumption process.
2. The message push platform of claim 1, wherein the message consumption process employs a message thread pool, the message thread pool comprising two or more message pull threads for pulling push messages in different storage locations in parallel from the message publish-subscribe component.
3. The message push platform of claim 1, wherein the message push process is further configured to: and determining a pushed list oriented to the pushed messages under the corresponding service application, which is pulled by the message consumption process, so as to respectively push the push content corresponding to each push message to each pushed party in the pushed list oriented to the push message.
4. The message push platform of claim 3, wherein the message push process is further configured to: if the overall length of the list of the pushed list exceeds the length of a preset list, converting the push messages corresponding to the pushed list into push message copies with specified quantity according to the detachable form of the pushed list, and storing the push message copies as new push messages into the message publishing and subscribing component;
the pushed list oriented by each pushed message copy under the corresponding service application is a target list splitting result matched with each list splitting result of the pushed message copy after the pushed list is split according to a split mode.
5. The message push platform of claim 1, further comprising: the message management process and a message update table corresponding to each preset storage position in the message publishing and subscribing component;
after the message pushing process generates the pushing content corresponding to the pushing message each time, recording the storage identifier set for the pushing message by the message publishing and subscribing component into a message updating table corresponding to the storage position of the pushing message in the message publishing and subscribing component;
and the message management process regularly reads the storage identification of the latest push message continuously completing consumption in each message update table and submits the storage identification to the message publishing and subscribing component.
6. The message push platform according to any one of claims 1 to 5, further comprising a message cache queue for caching push messages pulled by the message consuming process in parallel from the message publish-subscribe component;
the message pushing process comprises two or more message processing threads and is used for concurrently reading the pushing messages cached in the message caching queue and generating pushing contents corresponding to the pushing messages according to pushing characteristics under the service application associated with each pushing message.
7. The message push platform according to any of claims 1-5, further comprising a push front-end process and a message write process;
and the push front-end process generates corresponding push messages according to push requests under different service applications, calls the message writing process and writes the push messages into the message publishing and subscribing component.
8. The message pushing platform according to any one of claims 1 to 5, further comprising a content pushing component and a content issuing process, wherein the content pushing component stores pushing content corresponding to each pushing message generated by the message pushing process;
and the content issuing process issues each pushed content under the content pushing component to each pushed party in a pushed list facing the pushed message corresponding to the pushed content under the corresponding service application.
9. The message push platform of any of claims 1-5, wherein the message publish subscribe component is a Kafka messaging system.
10. A message pushing method applied to the message pushing platform of any one of claims 1 to 9, comprising:
the method comprises the steps that push messages stored by a Kafka message system facing different service applications are pulled in parallel through a message consumption process;
and concurrently generating push contents corresponding to each push message according to the push characteristics under the service application associated with each push message through the message push process.
11. The method of claim 10, wherein before generating the push content corresponding to each push message according to the push characteristics of the service application associated with each push message through the message push process, the method further comprises:
and calling a service processing process under the service application associated with each push message through the message push process, and determining the push characteristics under the service application associated with each push message.
12. The method of claim 10, further comprising, after pulling push messages stored by the Kafka messaging system towards different business applications in parallel through the message consuming process:
and determining a pushed list oriented to each push message under the corresponding service application through the message pushing process so as to respectively push the pushing content corresponding to each push message to each pushed party in the pushed list oriented to the push message.
13. The method of claim 12, further comprising:
if the overall length of the list of the pushed list oriented by the push message under the corresponding service application exceeds the length of a preset list, converting the push message into a specified number of push message copies according to the detachable form of the pushed list, storing the push message copies as new push messages into the Kafka message system, and continuously executing corresponding message push operation on the new push messages through the message consumption process and the message push process;
the pushed list oriented by each pushed message copy under the corresponding service application is a target list splitting result matched with each list splitting result of the pushed message copy after the pushed list is split according to a split mode.
14. The method according to claim 10, further comprising, after concurrently generating the push content corresponding to each push message:
recording a storage identifier set by the Kafka message system for the push message with the generated push content into a message update table corresponding to a storage position of the push message in the Kafka message system;
and regularly reading the storage identification of the latest push message continuously completing consumption in each message update table through a message management process, and submitting the storage identification to the Kafka message system.
15. The method according to any of claims 10-14, further comprising, before pulling in parallel, by the message consuming process, the push message stored by the Kafka messaging system towards different business applications:
and generating corresponding push messages according to push requests under different service applications, and writing the push messages into the Kafka message system.
16. The method according to any one of claims 10 to 14, further comprising, after concurrently generating the push content corresponding to each push message:
and respectively issuing each push content to each pushed party in a pushed list facing the push message corresponding to the push content under the corresponding service application.
17. A message push apparatus provided in the message push platform according to any one of claims 1 to 9, comprising:
the message pulling module is used for pulling the push messages stored by the Kafka message system facing different service applications in parallel through a message consumption process;
and the push content generation module is used for generating push contents corresponding to all the push messages through the message push process according to the push characteristics under the service application associated with each push message.
18. A server, characterized in that the server comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the message push method of any one of claims 10-16.
19. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the message push method according to any one of claims 10-16.
CN202010268157.8A 2020-04-08 2020-04-08 Message pushing platform, method, device, server and storage medium Pending CN111475759A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010268157.8A CN111475759A (en) 2020-04-08 2020-04-08 Message pushing platform, method, device, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010268157.8A CN111475759A (en) 2020-04-08 2020-04-08 Message pushing platform, method, device, server and storage medium

Publications (1)

Publication Number Publication Date
CN111475759A true CN111475759A (en) 2020-07-31

Family

ID=71750624

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010268157.8A Pending CN111475759A (en) 2020-04-08 2020-04-08 Message pushing platform, method, device, server and storage medium

Country Status (1)

Country Link
CN (1) CN111475759A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112311898A (en) * 2020-11-17 2021-02-02 深圳市欢太科技有限公司 Data processing method and device, server and computer readable storage medium
CN112448883A (en) * 2020-10-14 2021-03-05 苏宁云计算有限公司 Message pushing method and device, computer equipment and storage medium
CN112506915A (en) * 2020-10-27 2021-03-16 百果园技术(新加坡)有限公司 Application data management system, processing method and device and server
CN113747192A (en) * 2021-11-03 2021-12-03 腾讯科技(深圳)有限公司 Live broadcast control method and device, electronic equipment and storage medium
CN114329170A (en) * 2020-10-12 2022-04-12 北京达佳互联信息技术有限公司 Message pushing method and device, electronic equipment and storage medium
CN114745428A (en) * 2022-03-14 2022-07-12 许继电气股份有限公司 Converter station monitoring data model fast reading and writing method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106603689A (en) * 2016-12-26 2017-04-26 努比亚技术有限公司 Data processing method and device based on distributed message releasing and subscribing system
CN106953933A (en) * 2017-05-18 2017-07-14 腾讯科技(深圳)有限公司 A kind of information push method and device, electronic equipment
CN108337296A (en) * 2018-01-12 2018-07-27 深圳壹账通智能科技有限公司 Message push processing method, device, computer equipment and storage medium
CN108874562A (en) * 2018-06-21 2018-11-23 北京顺丰同城科技有限公司 Distributed high concurrent message queue supplying system
CN108965098A (en) * 2017-05-18 2018-12-07 北京京东尚科信息技术有限公司 Based on information push method, device, medium and the electronic equipment being broadcast live online
CN109640130A (en) * 2017-10-09 2019-04-16 武汉斗鱼网络科技有限公司 One kind starts broadcasting information push method, server and readable storage medium storing program for executing

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106603689A (en) * 2016-12-26 2017-04-26 努比亚技术有限公司 Data processing method and device based on distributed message releasing and subscribing system
CN106953933A (en) * 2017-05-18 2017-07-14 腾讯科技(深圳)有限公司 A kind of information push method and device, electronic equipment
CN108965098A (en) * 2017-05-18 2018-12-07 北京京东尚科信息技术有限公司 Based on information push method, device, medium and the electronic equipment being broadcast live online
CN109640130A (en) * 2017-10-09 2019-04-16 武汉斗鱼网络科技有限公司 One kind starts broadcasting information push method, server and readable storage medium storing program for executing
CN108337296A (en) * 2018-01-12 2018-07-27 深圳壹账通智能科技有限公司 Message push processing method, device, computer equipment and storage medium
CN108874562A (en) * 2018-06-21 2018-11-23 北京顺丰同城科技有限公司 Distributed high concurrent message queue supplying system

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114329170A (en) * 2020-10-12 2022-04-12 北京达佳互联信息技术有限公司 Message pushing method and device, electronic equipment and storage medium
CN112448883A (en) * 2020-10-14 2021-03-05 苏宁云计算有限公司 Message pushing method and device, computer equipment and storage medium
CN112506915A (en) * 2020-10-27 2021-03-16 百果园技术(新加坡)有限公司 Application data management system, processing method and device and server
CN112506915B (en) * 2020-10-27 2024-05-10 百果园技术(新加坡)有限公司 Application data management system, processing method and device and server
CN112311898A (en) * 2020-11-17 2021-02-02 深圳市欢太科技有限公司 Data processing method and device, server and computer readable storage medium
CN112311898B (en) * 2020-11-17 2023-07-18 深圳市欢太数字科技有限公司 Data processing method and device, server and computer readable storage medium
CN113747192A (en) * 2021-11-03 2021-12-03 腾讯科技(深圳)有限公司 Live broadcast control method and device, electronic equipment and storage medium
CN113747192B (en) * 2021-11-03 2022-02-22 腾讯科技(深圳)有限公司 Live broadcast control method and device, electronic equipment and storage medium
CN114745428A (en) * 2022-03-14 2022-07-12 许继电气股份有限公司 Converter station monitoring data model fast reading and writing method and device

Similar Documents

Publication Publication Date Title
CN111475759A (en) Message pushing platform, method, device, server and storage medium
US9917913B2 (en) Large message support for a publish-subscribe messaging system
US9838340B2 (en) Systems and methods for storing message data
CN110737536B (en) Message storage method and message reading method based on shared memory
US8495160B2 (en) System for controlling retention of data messages
CN110134648A (en) Log processing method, device, equipment, system and computer readable storage medium
JP5181184B2 (en) Apparatus and method for executing agent
CN111736907B (en) Data analysis method of self-adaptive low-delay memory computing engine
CN110688382A (en) Data storage query method and device, computer equipment and storage medium
CN113448752A (en) Index data acquisition method and device, electronic equipment and storage medium
WO2022211724A1 (en) Method and apparatus for transmitting messages, and device and storage medium thereof
CN112866339B (en) Data transmission method and device, computer equipment and storage medium
CN111416858B (en) Media resource processing platform, method, device and server
Yan et al. Efficient processing of very large graphs in a small cluster
CN113051221A (en) Data storage method, device, medium, equipment and distributed file system
CN111770054A (en) Interaction acceleration method and system for SMB protocol read request
CN111752941B (en) Data storage and access method and device, server and storage medium
CN111984198A (en) Message queue implementation method and device and electronic equipment
CN110955461A (en) Processing method, device and system of computing task, server and storage medium
CN116431290A (en) Job scheduling method, apparatus, device, medium and program product
CN110750362A (en) Method and apparatus for analyzing biological information, and storage medium
CN113986962A (en) Ranking list generation method, device, equipment and storage medium
CN113868249A (en) Data storage method and device, computer equipment and storage medium
CN112181737A (en) Message processing method and device, electronic equipment and medium
CN102253940B (en) Method and device for processing data by tree view

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200731

RJ01 Rejection of invention patent application after publication