WO2020177384A1

WO2020177384A1 - Method and apparatus for reporting and processing user message status of message pushing, and storage medium

Info

Publication number: WO2020177384A1
Application number: PCT/CN2019/117979
Authority: WO
Inventors: 乐志能
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-03-05
Filing date: 2019-11-13
Publication date: 2020-09-10
Also published as: CN109873904A; CN109873904B

Abstract

The present solution relates to research and development management and provided thereby are a method and apparatus for reporting and processing a user message status of message pushing, and a storage medium. The method comprises: each app sending report information to a message pushing background; the message pushing background storing the received report information in a local server memory queue (MQ) of the message pushing background, and the message pushing background feeding the received report information back to a user client; when the report information is stored in the local server MQ of the message pushing background, further starting up a plurality of worker threads to read in real time the report information stored in the MQ and store said information in an MQ storage component; then, the message pushing background reading each message in the report information in the MQ storage component, and verifying whether a reporting identity is legal; and if the reporting identity is legal, keeping the messages in an HDFS. The present solution accelerates storage speed and does not cause a sudden increase in access pressure for the message pushing background at a given moment.

Description

Method, device and storage medium for processing user message status reporting for message push

This application claims the priority rights of Chinese Patent Application No. 2019101649356 filed on March 5, 2019. The entire contents of the above cases are incorporated herein by reference.

Technical field

This application relates to research and development management, and specifically to a method, device and storage medium for processing user message status reporting for message push.

Background technique

Usually the user’s mobile phone receives the message push of the APP push will trigger the report information to be sent to the message push background. The reported information includes the message arrives on the mobile phone, the message is displayed in the notification bar or banner, click on the message, APP message notification shielding, APP uninstalled, etc. status information. After the message push background receives these status report requests, it first verifies whether the reported identity is legal, then decrypts the reported content, and then obtains which MsgID and which APP user reported the message status according to the decrypted content, and then saves it to the hard disk of the message push background In the relational database above, this is an external storage. The inventor realized that when the user scale reaches 100 million and simultaneously pushes messages to users, APP users will simultaneously report a large number of message statuses to the message push background, and an average of 2-3 statuses of a push message must be reported to the message push change In the background, the message push background receives the reported information and saves it to a relational database (such as mysql, oracle) for storage, which will result in slow service processing and eventually unavailability of the service.

Summary of the invention

To solve the above technical problems, this application provides a user message status report processing method for message push, which is applied to an electronic device, and includes:

After the message push background pushes messages to the APPs on each mobile terminal, the message push background receives the reported information sent by each APP. The reported information includes at least the arrival of the message on the mobile terminal, the message displayed in the notification bar or banner, user clicks, and APP Whether the message notification is blocked and whether the APP has been uninstalled;

The message push background stores the received report information in the local server memory queue of the message push background, and the message push background reports to each mobile terminal that the reported information has been received;

While storing the reported information in the memory queue of the local server in the message push background, the message push background also starts multiple workers threads to read the reported information stored in the memory queue in real time and store it in the MQ storage component;

The message push background reads each message in the reported information in the consumer MQ storage component, verifies whether the reported identity is legal, and decrypts the content of the message. If the reported identity is legal, the message is stored in HDFS.

The present application also provides an electronic device, the electronic device comprising: a memory and a processor, the memory stores a user message status report processing program for message push, and the user message status report processing program for message push is processed by the The following steps are implemented when the device is executed:

After the message push background pushes messages to the APPs on each mobile terminal, each APP sends report information to the message push background, and the report information includes multiple pieces of information returned by the APP from which the message is pushed from each mobile terminal. The report The information includes at least the arrival of the message on the mobile terminal, the display of the message in the notification bar or banner, user clicks, whether the APP message notification is blocked, and whether the APP has been uninstalled;

The message push background directly stores the received report information in the local server memory queue of the message push background, and then the message push background sends feedback to each mobile terminal that the reported information has been received;

Then the message push background reads each message in the reported information in the consumer MQ storage component, and verifies whether the reported identity is legal, and decrypts the message content. If the reported identity is legal, the message is stored in HDFS.

This application also provides a user message status report processing device for message push, including:

The report message receiving module, after the message push background pushes the message to the APP on each mobile terminal, the message push background receives the report information sent by each APP, and the report information includes at least the message reached the mobile terminal, the message is displayed in the notification bar or banner, User clicks, whether the APP message notification is blocked, and whether the APP has been uninstalled;

The report message memory queue storage module is used for the message push background to directly store the received report information in the local server memory queue of the message push background, and then the message push background reports to each mobile terminal that the reported information has been received;

The memory queue information reading module is used to store the reported information in the local server memory queue of the message push background. At the same time, the message push background also starts multiple workers thread to read the reported information stored in the memory queue in real time and store it in MQ Storage components;

The message middleware reading module is used for the message push background to read each message in the reported information in the consumer MQ storage component, and verify whether the reported identity is legal, and decrypt the message content. If the reported identity is legal, keep the message To HDFS.

The present application also provides a computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, The user message status report processing method for message push as described above is implemented.

In this application, the message status information is processed asynchronously. The message status information is first stored in the server memory queue. The storage speed is fast. In addition, while storing it in the memory queue, it also uses multiple worker threads to read the memory queue in real time. The reported information is stored in the MQ storage component, which will not cause a surge in access pressure to the message push background at a certain moment. The message push background then reads each message in the reported information in the consumer MQ storage component, and saves the message in HDFS after the reported identity is legal. When a large number of clients need to report status to the message push background, this application speeds up the storage speed and the client responds quickly.

Description of the drawings

By describing its embodiments in conjunction with the following drawings, the above-mentioned features and technical advantages of the present application will become clearer and easier to understand.

FIG. 1 is a schematic flowchart of a method for processing user message status reporting in message push according to an embodiment of the present application;

2 is a schematic diagram of the hardware architecture of an electronic device according to an embodiment of the present application;

Fig. 3 is a block diagram of a user message status report processing program for message push in an embodiment of the present application.

detailed description

Hereinafter, embodiments of the method, device, and storage medium for reporting user message status for message push in this application will be described with reference to the accompanying drawings. A person of ordinary skill in the art may realize that the described embodiments can be modified in various different ways or combinations thereof without departing from the spirit and scope of the present application. Therefore, the drawings and descriptions are illustrative in nature and are not used to limit the scope of protection of the claims. In addition, in this specification, the drawings are not drawn to scale, and the same reference numerals denote the same parts.

FIG. 1 is a schematic flowchart of a method for processing user message status reporting for message push provided by an embodiment of the application. The method is applied to an electronic device and includes the following steps:

Step S10, after the message push background pushes messages to the APPs on each mobile terminal, each APP sends report information to the message push background, and the report information includes multiple pieces of information returned by the APPs that have pushed messages from each mobile terminal. The reported information includes at least the arrival of the message on the mobile terminal, the display of the message in the notification bar or banner, the user clicks, whether the APP message notification is blocked, and whether the APP has been uninstalled.

Step S20: The message push background directly stores the received report information in the local server memory queue of the message push background, and then the message push background feeds back the received report information to each mobile terminal.

Step S30: While storing the reported information in the local server memory queue of the message push background, the message push background also starts multiple workers threads (worker threads) to read the reported information stored in the memory queue in real time and store it in the MQ storage component (Message middleware). In worker Thread mode, each message in the reported information is taken out one by one for processing.

When all the messages are read, the worker thread will wait for new messages. The worker thread stores the read report information in the MQ storage component, such as kafka (distributed, partitioned, and reproducible submission log service). Kafka categorizes reported messages according to topics when they are saved. Topics can be considered as a type of message. Each topic is divided into multiple partitions, and each partition is an append.log file at the storage level. In Kafka, the person who sends the message is called the Producer, where the worker thread is equivalent to the producer. The Producer publishes the reported message to the specified topic, and the Producer can also determine which partition the reported message belongs to. Any message posted to the partition will be directly appended to the end of the log file. The position of each message in the file is called the offset. The offset is a long number, and the offset uniquely marks a message.

Step S40, then the message push background reads each message in the reported information in the consumer MQ storage component, and verifies whether the reported identity is legal, and decrypts the message content. If the reported identity is legal, the message is saved to HDFS (distributed File system).

Preferably, a big data platform is used to read each message in the reported information in the consumption kafka, which can speed up the processing speed.

Further, the method for the message push background to directly store the received report information in the local server memory queue of the message push background is: through Map msgstats=new HashMap(), the reported information status object is placed in the map memory collection of the local server memory Among them, HashMap internally stores data through the array Entry[]. HashMap uses the key-value interface of Map. The definition of Entry class is:

class Entry{

Object key

Object value

Entry next;

}

If the key corresponding to the key-value pair added to the HashMap already exists in the HashMap, find the key-value pair and replace the old value with the new value. If the key corresponding to the key-value pair to be added to the HashMap is not in the HashMap , Then add it to the corresponding linked list.

Further, for the producer side of Kafka, set the transmission buffer threshold, buffer the messages, and send them to Kafka in batches after the number of messages reaches the transmission buffer threshold;

For the consumer side of Kafka, set the consumption buffer threshold. When the message in Kafka reaches the consumption buffer threshold, the message push background will fetch multiple messages in batches.

In an optional embodiment, data cleaning is performed using an ETL (data cleaning) cleaning program to remove dirty data, where the dirty data includes duplicate data, incomplete data, invalid data, and inconsistent data, wherein, before data cleaning First, the messages are clustered according to repeated data, incomplete data, invalid data, and inconsistent data.

In an optional embodiment, repeated data cleaning includes judging whether it is duplicated and deleting duplicate data, wherein a multi-segment comparison method is used to sequentially compare the same length fields in the same position of no less than 3 two messages. To determine whether the two messages are duplicate data. Specifically, judging whether to repeat can be based on extracting a segment of the same position of the two messages, and comparing whether the two segments are exactly the same, if they are exactly the same, extract the two messages at a certain interval (for example, one character) Another field in the same position of the two messages, and compare whether the two fields are exactly the same. If they are exactly the same, extract another field in the same position of the two messages at a certain interval and compare them further. If they are the same, the two fields are considered to be the same. Segment messages are repeated data. If they are not the same, they are deemed not to be repeated data. Wherein, preferably, the same positions of the messages extracted three times are the head of the message field, the tail of the message field, and the middle of the message field, respectively.

Further, for data stored in the form of key-value pairs, the above method is first used to determine whether the keys are completely consistent, and then whether the values are completely consistent. Among them, the key-value pair is presented in the form of key=value. Here it means that the key is first judged whether it is completely consistent, and then the value is judged whether it is completely consistent, so as to judge whether the message is a duplicate message.

In an optional embodiment, before the two messages are compared in a multi-segment comparison method, the cosine similarity is calculated according to the key-value pair vector composed of the key and value of the two messages, and the preset similarity is calculated The threshold is used to determine the degree of similarity. If it is higher than the threshold of similarity, it is considered a duplicate message. Specifically, first cluster all the messages in the reported information, for example, classify all messages from the same mobile terminal into one category, and then group the key and value of each message into a key-value pair vector to calculate two messages If the cosine similarity of the key-value pair vector is higher than the similarity threshold, the same length fields at the same position of the two messages at no less than 3 are used to compare in turn to determine whether the two messages are duplicate data. If the similarity is lower than the similarity threshold, it is considered not to be duplicate data.

For example, the vector of key-value pairs of message A and message B are as follows:

Among them, d(A, B) represents the cosine similarity of message A and message B;

key _Ai is the i-th component of the key vector representing message A;

value _Ai is the i-th component of the value vector representing message A;

key _Bi is the i-th component of the key vector representing message B;

value _Bi is the i-th component of the value vector representing the message B;

n represents the number of key-value pairs of message A and message B.

After calculating the cosine similarity of the two messages, if the similarity is higher than the similarity threshold, no less than 3 two messages with the same length at the same position are compared in turn to determine whether the two messages are Duplicate data. Can speed up the speed of message checking. If the similarity is lower than the similarity threshold, it is considered not to be duplicate data.

Refer to FIG. 2, which is a schematic diagram of the hardware architecture of an embodiment of the electronic device of the present application. In this embodiment, the electronic device 2 is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions. For example, it can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers). As shown in FIG. 2, the electronic device 2 at least includes, but is not limited to, a memory 21, a processor 22, and a network interface 23 that can be communicatively connected to each other through a system bus. Wherein: the memory 21 includes at least one type of computer non-volatile readable storage medium, the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random Access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. . In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or a memory of the electronic device 2. In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, for example, a plug-in hard disk equipped on the electronic device 2, a smart media card (SMC), a secure digital (Secure Digital, SD) card, flash card (Flash Card), etc. Of course, the memory 21 may also include both the internal storage unit of the electronic device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store the operating system and various application software installed in the electronic device 2, for example, the user message status report processing program code of the message push, etc. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

The processor 22 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 22 is generally used to control the overall operation of the electronic device 2, for example, to perform data interaction or communication-related control and processing with the electronic device 2. In this embodiment, the processor 22 is configured to run the program code or processing data stored in the memory 21, for example, to run the user message status report processing program of the message push.

The network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the electronic device 2 and other electronic devices. For example, the network interface 23 is used to connect the electronic device 2 with a push platform through a network, and establish a data transmission channel and a communication connection between the electronic device 2 and the push platform. The network may be Intranet, Internet, Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network , Bluetooth (Bluetooth), Wi-Fi and other wireless or wired networks.

Optionally, the electronic device 2 may also include a display, and the display may also be called a display screen or a display unit. In some embodiments, it may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) display, etc. The display is used for displaying information processed in the electronic device 2 and for displaying a visualized user interface.

It should be pointed out that FIG. 2 only shows the electronic device 2 with components 21-23, but it should be understood that it is not required to implement all of the illustrated components, and more or fewer components may be implemented instead.

The memory 21 containing a readable storage medium may include an operating system, a user message status report processing program 50 for message push, and the like. The processor 22 implements the following steps when executing the user message status report processing program 50 of the message push in the memory 21:

Step S30, while storing the reported information in the memory queue of the local server in the message push background, multiple worker threads are also started to read the reported information stored in the memory queue in real time and store it in the MQ storage component. In worker Thread mode, each message in the reported information is taken out one by one for processing.

In step S40, the message push background reads each message in the reported information in the consumer MQ storage component, verifies whether the reported identity is legal, and decrypts the message content. If the reported identity is legal, the message is stored in HDFS.

In this embodiment, the user message status report processing program of the message push stored in the memory 21 may be divided into one or more program modules, and the one or more program modules are stored in the memory 21, and It can be executed by one or more processors (the processor 22 in this embodiment) to complete the application. For example, FIG. 3 shows a schematic diagram of the program modules of the user message status report processing program of the message push. In this embodiment, the user message status report processing program 50 of the message push can be divided into a report message receiving module 501, Report message memory queue storage module 502, memory queue information reading module 503, and message middleware reading module 504. Among them, the program module referred to in this application refers to a series of computer program instruction segments that can complete specific functions, and is more suitable than a program to describe the execution process of the user message status report processing program of the message push in the electronic device 2 . The following description will specifically introduce the specific functions of the program modules.

The report message receiving module 501 is configured to receive the report message when each APP sends the report information to the message push background after pushing the message to the APP on each mobile terminal in the message push background. The reported information includes multiple pieces of information returned by the APP from which the message is pushed from each mobile terminal. The reported information includes at least the arrival of the message on the mobile terminal, the display of the message in the notification bar or banner, the user click, and whether the APP message notification is Blocking, whether the APP has been uninstalled.

The reported message memory queue storage module 502 is used to directly store the received reported information in the local server memory queue of the message push background after the message push background, and then the message push background feedbacks the received report information to each mobile terminal.

The memory queue information reading module 503 is used to store the reported information in the memory queue of the local server in the message push background, and also start multiple worker threads to read the reported information stored in the memory queue in real time and store it in the MQ storage component . In worker Thread mode, each message in the reported information is taken out one by one for processing.

The message middleware reading module 504 is used to read each message in the reported information in the consumer MQ storage component, and verify whether the reported identity is legal, and decrypt the message content. If the reported identity is legal, the message is stored in HDFS .

Further, the report message memory queue storage module 502 directly stores the received report information in the local server memory queue of the message push background by using Map msgstats=new HashMap() to put the report information status object into the map memory of the local server memory In the collection, the inside of HashMap stores data through the array Entry[]. HashMap uses the key-value interface of Map. The definition of Entry class is:

class Entry{

Object key

Object value

Entry next;

}

Further, it also includes a threshold comparison module 505. For the producer side of Kafka, a transmission buffer threshold is set, and messages are buffered. After the number of messages reaches the transmission buffer threshold, they are sent to Kafka in batches;

In an optional embodiment, a data cleaning module 506 is further included. The data cleaning module 506 uses an ETL (data cleaning) cleaning program to clean data to remove dirty data, where the dirty data includes duplicate data, incomplete data, and invalid data. Data and inconsistent data. Before data cleaning, messages are clustered according to repeated data, incomplete data, invalid data, and inconsistent data.

In an optional embodiment, it further includes a repeated data judging module 507. The repeated data judging module 507 is used for judging whether the data is repeated and deleting the repeated data. Among them, a multi-segment comparison method is adopted, and two of no less than 3 The same length fields in the same position of the message are compared sequentially to determine whether the two messages are duplicate data. Specifically, judging whether to repeat can be based on extracting a segment of the same position of the two messages, and comparing whether the two segments are exactly the same, if they are exactly the same, extract the two messages at a certain interval (for example, one character) Another field in the same position of the two messages, and compare whether the two fields are exactly the same. If they are exactly the same, extract another field in the same position of the two messages at a certain interval and compare them further. If they are the same, the two fields are considered to be the same. Segment messages are repeated data. If they are not the same, they are deemed not to be repeated data. Wherein, preferably, the same positions of the messages extracted three times are the head of the message field, the tail of the message field, and the middle of the message field, respectively.

In an optional embodiment, the repeated data judging module 507 also calculates the cosine similarity based on the key-value pair vector composed of the key and value of the two messages. Before comparing the two messages in a multi-segment comparison method, first The preset similarity threshold is used to determine the degree of similarity. If it is higher than the similarity threshold, it is considered as a duplicate message. Specifically, first cluster all the messages in the reported information, for example, classify all messages from the same mobile terminal into one category, and then group the key and value of each message into a key-value pair vector to calculate two messages If the cosine similarity of the key-value pair vector is higher than the similarity threshold, no less than 3 two messages with the same length at the same position are compared in sequence to determine whether the two messages are duplicate data. Can speed up the speed of message checking. If the similarity is lower than the similarity threshold, it is considered not to be duplicate data.

The report message receiving module 501, after the message push background pushes messages to the APPs on each mobile terminal, the message push background receives the report information sent by each APP, and the report information includes at least the message reached the mobile terminal and the message is displayed in the notification bar or banner , User clicks, whether the APP message notification is blocked, and whether the APP has been uninstalled;

The report message memory queue storage module 502 is used for the message push background to directly store the received report information in the local server memory queue of the message push background, and then the message push background reports to each mobile terminal that the report information has been received;

The memory queue information reading module 503 is used to store the reported information in the memory queue of the local server in the message push background. At the same time, the message push background also starts multiple workers threads to read the reported information stored in the memory queue in real time and store it in MQ storage component;

The message middleware reading module 504 is used for the message push background to read each message in the reported information in the consumer MQ storage component, verify whether the reported identity is legal, and decrypt the message content. If the reported identity is legal, the message Keep it in HDFS.

In addition, the embodiment of the present application also proposes a computer non-volatile readable storage medium. The computer non-volatile readable storage medium may be a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory ( ROM), erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, etc., or any combination of several. The computer non-volatile readable storage medium includes a user message status report processing program for message push, etc. The user message status report processing program 50 for message push is executed by the processor 22 to implement the following operations:

Step S30, while storing the reported information in the memory queue of the local server in the message push background, the message push background also starts multiple workers threads to read the reported information stored in the memory queue in real time and store it in the MQ storage component. In worker Thread mode, each message in the reported information is taken out one by one for processing.

In step S40, the message push background reads each message in the reported information in the consumer MQ storage component, and verifies whether the reported identity is legal, and decrypts the message content. If the reported identity is legal, the message is stored in HDFS.

The specific implementation manner of the computer non-volatile readable storage medium of the present application is substantially the same as the specific implementation manner of the user message status report processing method for message push and the electronic device 2 described above, and will not be repeated here.

The foregoing descriptions are only preferred embodiments of the application, and are not intended to limit the application. For those skilled in the art, the application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

A user message status report processing method for message push, applied to an electronic device, and is characterized in that it includes:

After the message push background pushes messages to the APPs on each mobile terminal, the message push background receives the reported information sent by each APP. The reported information includes at least the arrival of the message on the mobile terminal, the message displayed in the notification bar or banner, user clicks, and APP Whether the message notification is blocked and whether the APP has been uninstalled;

The message push background stores the received report information in the local server memory queue of the message push background, and then the message push background sends feedback to each mobile terminal that the reported information has been received;

While storing the reported information in the memory queue of the local server in the message push background, the message push background also starts multiple workers threads to read the reported information stored in the memory queue in real time and store it in the MQ storage component;

The message push background reads each message in the reported information in the consumer MQ storage component, verifies whether the reported identity is legal, and decrypts the content of the message. If the reported identity is legal, the message is stored in HDFS.
The user message status report processing method for message push according to claim 1, characterized in that:

The method for the message push background to store the received report information in the memory queue of the local server in the message push background is: through Map msgstats=new HashMap(), the reported information status object is placed in the map memory collection of the local server memory.

Among them, HashMap internally stores data through the array Entry[], and HashMap uses Map’s key-value key-value interface,

If the key corresponding to the key-value pair added to the HashMap already exists in the HashMap, find the key-value pair and replace the old value with the new value. If the key corresponding to the key-value pair to be added to the HashMap is not in the HashMap , Then add it to the corresponding linked list.
The user message status report processing method for message push according to claim 1, characterized in that:

For the producer side of the worker thread, set the transmission buffer threshold, buffer the messages, and send them to the MQ storage component in batches after the number of messages reaches the transmission buffer threshold;

For the consumer side of the worker thread, set the consumption buffer threshold. When the message in the MQ storage component reaches the consumption buffer threshold, the message push background will fetch multiple messages in batches.
The user message status report processing method for message push according to claim 1, characterized in that:

The ETL cleaning program is used for data cleaning to remove dirty data. The dirty data includes duplicate data, incomplete data, invalid data, and inconsistent data. Before data cleaning, the message is classified as duplicate data, incomplete data, and invalid data. , Clustering inconsistent data.
The user message status report processing method for message push according to claim 4, characterized in that:

First calculate the cosine similarity according to the key-value pair vector composed of the key and value of the two messages, and judge the similarity according to the preset similarity threshold. If it is higher than the similarity threshold, then use the message field of the two messages. The header, the tail of the message field, and the same length fields in the middle of the message field are compared in sequence to finally determine whether the two messages are duplicate data.
A user message status report processing device for message push, which is characterized in that it comprises:

The report message receiving module, after the message push background pushes the message to the APP on each mobile terminal, the message push background receives the report information sent by each APP, and the report information includes at least the message reached the mobile terminal, the message is displayed in the notification bar or banner, User clicks, whether the APP message notification is blocked, and whether the APP has been uninstalled;

The report message memory queue storage module is used for the message push background to directly store the received report information in the local server memory queue of the message push background, and then the message push background reports to each mobile terminal that the reported information has been received;

The memory queue information reading module is used to store the reported information in the local server memory queue of the message push background. At the same time, the message push background also starts multiple workers thread to read the reported information stored in the memory queue in real time and store it in MQ Storage components;

The message middleware reading module is used for the message push background to read each message in the reported information in the consumer MQ storage component, and verify whether the reported identity is legal, and decrypt the message content. If the reported identity is legal, keep the message To HDFS.
The user message status report processing device for message push according to claim 6, characterized in that it further comprises a threshold comparison module,

For the producer side of the worker thread, set the transmission buffer threshold and buffer the messages. After the number of messages reaches the transmission buffer threshold, they will be sent to the MQ storage component in batches;

For the consumer side of the worker thread, set the consumption buffer threshold. When the message in the MQ storage component reaches the consumption buffer threshold, the message push background will fetch multiple messages in batches.
The user message status report processing device for message push according to claim 6, characterized in that it further comprises a data cleaning module, which is used to clean data by using an ETL cleaning program to remove dirty data, wherein the dirty data includes duplicates Data, incomplete data, invalid data, and inconsistent data. Before data cleaning, the message is clustered according to repeated data, incomplete data, invalid data, and inconsistent data.
The user message status report processing device for message push according to claim 8, characterized in that it further comprises a repeated data judging module, which judges whether to repeatedly use a multi-segment comparison method, and the two messages in no less than 3 places are identical. The positions of the same length fields are compared sequentially to determine whether the two messages are duplicate data.
According to the user message status report processing device for message push according to claim 9, the cosine similarity is calculated according to the key-value pair vector composed of the key and value of the two messages, and the similarity is judged according to the preset similarity threshold If it is higher than the similarity threshold, the header of the message field, the tail of the message field, and the same length field in the middle of the message field of the two messages are compared in sequence to finally determine whether the two messages are duplicate data.
An electronic device, characterized in that the electronic device includes a memory and a processor, the memory stores a user message status report processing program for message push, and the user message status report processing program for message push is processed by the The following steps are implemented when the device is executed:

After the message push background pushes messages to the APPs on each mobile terminal, the message push background receives the reported information sent by each APP. The reported information includes at least the arrival of the message on the mobile terminal, the message displayed in the notification bar or banner, user clicks, and APP Whether the message notification is blocked and whether the APP has been uninstalled;

The message push background directly stores the received report information in the local server memory queue of the message push background, and then the message push background sends feedback to each mobile terminal that the reported information has been received;

While storing the reported information in the memory queue of the local server in the message push background, the message push background also starts multiple workers threads to read the reported information stored in the memory queue in real time and store it in the MQ storage component;

Then the message push background reads each message in the reported information in the consumer MQ storage component, and verifies whether the reported identity is legal, and decrypts the message content. If the reported identity is legal, the message is stored in HDFS.
The electronic device according to claim 11, wherein the user message status report processing program of the message push is also implemented when the processor is executed:

For the producer side of the worker thread, set the transmission buffer threshold, buffer the messages, and send them to the MQ storage component in batches after the number of messages reaches the transmission buffer threshold;

For the consumer side of the worker thread, set the consumption buffer threshold. When the message in the MQ storage component reaches the consumption buffer threshold, the message push background will fetch multiple messages in batches.
The electronic device according to claim 11, wherein the user message status report processing program of the message push is also implemented when the processor is executed:

The ETL cleaning program is used for data cleaning to remove dirty data. The dirty data includes duplicate data, incomplete data, invalid data, and inconsistent data. Before data cleaning, the message is classified as duplicate data, incomplete data, and invalid data. , Clustering inconsistent data.
The electronic device according to claim 13, wherein the user message status report processing program of the message push is also implemented when the processor is executed:

The cleaning of duplicate data includes judging whether it is duplicated and deleting duplicate data. Among them, judging whether to repeatedly use multiple comparisons, and compare the same length fields in the same position of two messages at no less than 3 to determine the two. Whether the message is duplicate data.
According to the electronic device of claim 14, the user message status report processing program of the message push is further implemented when being executed by the processor:

First calculate the cosine similarity according to the key-value pair vector composed of the key and value of the two messages, and judge the similarity according to the preset similarity threshold. If it is higher than the similarity threshold, then use the message field of the two messages. The header, the tail of the message field, and the same length fields in the middle of the message field are compared in sequence to finally determine whether the two messages are duplicate data.
A computer non-volatile readable storage medium, wherein the computer non-volatile readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, The user message status report processing method for message push of claim 1 is realized.
The computer non-volatile readable storage medium according to claim 16, wherein when the program instructions are executed by the processor, they further implement:

For the producer side of the worker thread, set the transmission buffer threshold, buffer the messages, and send them to the MQ storage component in batches after the number of messages reaches the transmission buffer threshold;

For the consumer side of the worker thread, set the consumption buffer threshold. When the message in the MQ storage component reaches the consumption buffer threshold, the message push background will fetch multiple messages in batches.
The computer non-volatile readable storage medium according to claim 16, wherein when the program instructions are executed by the processor, they further implement:

The ETL cleaning program is used for data cleaning to remove dirty data. The dirty data includes duplicate data, incomplete data, invalid data, and inconsistent data. Before data cleaning, the message is classified as duplicate data, incomplete data, and invalid data. , Clustering inconsistent data.
The computer non-volatile readable storage medium according to claim 16, wherein when the program instructions are executed by the processor, they further implement:

The cleaning of duplicate data includes judging whether it is duplicated and deleting duplicate data. Among them, judging whether to repeatedly use multiple comparisons, and compare the same length fields in the same position of two messages at no less than 3 to determine the two. Whether the message is duplicate data.
The computer non-volatile readable storage medium according to claim 19, wherein when the program instruction is executed by the processor, it further implements:

First calculate the cosine similarity according to the key-value pair vector composed of the key and value of the two messages, and judge the similarity according to the preset similarity threshold. If it is higher than the similarity threshold, then use the message field of the two messages. The header, the tail of the message field, and the same length fields in the middle of the message field are compared in sequence to finally determine whether the two messages are duplicate data.