CN111669313B - Method for storing and synchronizing instant communication system message - Google Patents
Method for storing and synchronizing instant communication system message Download PDFInfo
- Publication number
- CN111669313B CN111669313B CN202010476297.4A CN202010476297A CN111669313B CN 111669313 B CN111669313 B CN 111669313B CN 202010476297 A CN202010476297 A CN 202010476297A CN 111669313 B CN111669313 B CN 111669313B
- Authority
- CN
- China
- Prior art keywords
- subscription
- message
- sub
- message processing
- processing service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/21—Monitoring or handling of messages
- H04L51/216—Handling conversation history, e.g. grouping of messages in sessions or threads
Abstract
The invention discloses a method for storing and synchronizing messages of an instant messaging system, and belongs to the technical field of instant messaging. The method comprises the following steps: after a message main program receives a message storage request and a message synchronization request, the main program calls a message subscriber inquiry interface, the inquiry interface inquires a subscription id array of a user, the subscription id array is divided into 4 parts, and 4 threads are used for concurrently inquiring messages corresponding to the 4 subscription ids; the main program traverses the subscription id array, and calls a subscription id configurator to obtain the message processing service and the sub-table corresponding to the subscription id; and calling the message processing service by the main program to inquire the message corresponding to the subscription id from the corresponding sub-table, and merging all the subscription messages and returning after finishing the processing of the 4 threads. The invention can effectively reduce the repeated storage amount of the message.
Description
Technical Field
The invention belongs to the technical field of instant messaging, and particularly relates to a method for storing and synchronizing messages of an instant messaging system.
Background
The traditional large instant messaging message storage algorithm generally uses a write diffusion mode and stores the hash of the user id needing to receive the message in a database and table. This approach consumes a large amount of memory space and may store messages repeatedly. And the database partitioning algorithm is carried out in the application service, and a large number of different database connections can be generated by the same service. The partial algorithm design can distinguish the storage of the message read diffusion and the message write diffusion, but the control of a definite threshold value is not used, the single chat message is subjected to the write diffusion storage, and the group chat message is subjected to the read diffusion storage, so that the query times of the synchronous message can be increased during the synchronous message.
Disclosure of Invention
The invention aims to provide a method for storing and synchronizing messages of an instant messaging system, which can effectively reduce the repeated message storage amount.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a method for storing and synchronizing messages in an instant communication system comprises the following steps:
s1, when the main program of the message receives the request of message storage, the main program calls the subscription object creating interface of the message subscriber, the subscriber generates a subscription message object according to the message type, and the subscription message object contains subscription id, diffusion type and message content;
s2, calling a subscription id configurator by the main program according to the subscription id of the subscription message object to acquire a message processing service address and a sub-table name corresponding to the subscription id, and returning the message processing service address and the sub-table name by the subscription id configurator according to the rule;
s3, the main program calls the message processing service according to the message service address to store the subscription message object into the corresponding sub-table;
s4, when the main program receives the message synchronization request, the main program calls a message subscriber inquiry interface, the inquiry interface inquires the subscription id array of the user, the subscription id array is divided into 4 parts, and 4 threads are used for inquiring messages corresponding to the 4 subscription ids;
s5, traversing the subscription id array by the main program, and calling a subscription id configurator to acquire the message processing service and the sub-table corresponding to the subscription id;
and S6, calling the message processing service by the main program to inquire the message corresponding to the subscription id from the corresponding sub-table, and merging all the subscription messages and returning after the 4 threads are processed.
Further, in step S1, the message type includes single chat information and group chat information, and the user id of the sending message and the user id of the receiving message are used as subscription ids in the case of the single chat information, and the group id is used as subscription ids in the case of the group chat message.
Further, the subscription id configurator structure includes:
subscription id type configuration, which is used for configuring the names of message processing services with different subscription id types, the maximum number of the message processing services, the stored sub-table names and the maximum number of the stored sub-tables;
the message processing service instance configuration is used for recording the total number of the subscription ids corresponding to each message processing service currently;
the message processing service sub-table instance configuration is used for recording the total number of the subscription ids currently corresponding to each sub-table in each message processing service; and
and the subscription id maps the position configuration, and the position configuration is used for recording the message processing service instance and the sub-table instance corresponding to the subscription id.
Further, the querying and generating of the subscription id mapping location comprises the following steps:
1) the main program inquires a corresponding message processing service instance and a sub-table instance in the configuration of the subscription id mapping position according to the subscription id, if the message processing service instance and the sub-table instance are found, the return is made, and if the message processing service instance and the sub-table instance are not found, the next step is carried out;
2) the subscription id configurator finds the name of the corresponding message processing service, the maximum number of the message processing services, the stored sub-table name and the stored maximum number of the sub-tables in the subscription id type configuration according to the type of the subscription id;
3) according to the message processing service name, finding a corresponding latest message processing service instance in the message processing service instance configuration;
4) according to the message processing service instance, finding a corresponding latest sub-table instance in the message processing service sub-table instance configuration;
5) and the main program returns the corresponding message processing service instance and the sub-table instance of the subscription id, and a record is newly added in the configuration of the mapping position of the subscription id.
Further, in the step 3), if the number of the latest message processing service instances processed is greater than or equal to the configuration number, adding a new message processing service instance, and if the number of the latest message processing service instances processed is less than the configuration number, returning to the current message processing service instance, and adding 1 to the number of the current message processing service instances processed; if the corresponding latest message processing service instance cannot be found, a new message processing service instance is created.
Further, in the step 4), if the processing number of the sub-table instances is greater than or equal to the configured number, adding one sub-table instance, if the processing number of the sub-table instances is less than the configured number, returning to the current sub-table instance, adding 1 to the processing number of the current sub-table instance, and if the sub-table instance cannot be found, creating a new sub-table instance.
Due to the adoption of the technical scheme, the invention has the following beneficial effects:
1. the invention can distinguish according to different message types, designs the corresponding subscription id, uniformly uses the subscription id to synchronize or store the messages, enables the messages with different subscription ids to be processed by different services in a database-sharing and table-sharing mode, and effectively reduces the repeated storage amount of the messages.
2. The invention can dynamically expand the capacity of the message processing service and divide the database and the table, thereby ensuring that the system can still efficiently store and synchronize the message when the service volume is increased.
3. The invention can use the number of the group chat users as the weight to determine whether to use a read diffusion mode or a write diffusion mode to store the message of the user, and use a multi-thread query mode to reduce the message synchronization time.
Drawings
FIG. 1 is a flow chart of a method for storing and synchronizing messages in an instant messaging system according to the present invention;
FIG. 2 is a flowchart of a system message storage method according to embodiment 1;
fig. 3 is a flowchart of a system message synchronization method in embodiment 1.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
The operation of the present invention has several major constructs:
1. the message subscriber: and the system is responsible for providing a uniform message storage and synchronization interface for user messages.
2. A subscription id configurator: and the system is responsible for mapping different subscription ids to different message processing services through a certain rule and a table in which messages need to be stored.
3. Message processing service: storing messages to corresponding base tables according to rules, wherein each service corresponds to one database, and providing a restful interface can be horizontally expanded
As shown in fig. 1, a method for storing and synchronizing messages in an instant messaging system includes the following steps:
s1, when the main program of the message receives the request of message storage, the main program calls the subscription object creating interface of the message subscriber, the subscriber generates a subscription message object according to the message type, and the subscription message object contains subscription id, diffusion type and message content;
s2, calling a subscription id configurator by the main program according to the subscription id of the subscription message object to acquire a message processing service address and a sub-table name corresponding to the subscription id, and returning the message processing service address and the sub-table name by the subscription id configurator according to the rule;
s3, the main program calls the message processing service according to the message service address to store the subscription message object into the corresponding sub-table;
s4, when the main program receives the message synchronization request, the main program calls a message subscriber inquiry interface, the inquiry interface inquires the subscription id array of the user, the subscription id array is divided into 4 parts, and 4 threads are used for inquiring messages corresponding to the 4 subscription ids;
s5, traversing the subscription id array by the main program, and calling a subscription id configurator to acquire the message processing service and the sub-table corresponding to the subscription id;
and S6, calling the message processing service by the main program to inquire the message corresponding to the subscription id from the corresponding sub-table, and merging all the subscription messages and returning after the 4 threads are processed.
Further, in step S1, the message type includes single chat information and group chat information, and the user id of the sending message and the user id of the receiving message are used as subscription ids in the case of the single chat information, and the group id is used as subscription ids in the case of the group chat message.
The subscription-id configurator structure includes:
subscription id type configuration, which is used for configuring the names of message processing services with different subscription id types, the maximum number of the message processing services, the stored sub-table names and the maximum number of the stored sub-tables;
the message processing service instance configuration is used for recording the total number of the subscription ids corresponding to each message processing service currently;
the message processing service sub-table instance configuration is used for recording the total number of the subscription ids currently corresponding to each sub-table in each message processing service; and
and the subscription id maps the position configuration, and the position configuration is used for recording the message processing service instance and the sub-table instance corresponding to the subscription id.
The inquiry and generation of the subscription id mapping position comprises the following steps:
1) the main program inquires a corresponding message processing service instance and a sub-table instance in the configuration of the subscription id mapping position according to the subscription id, if the message processing service instance and the sub-table instance are found, the return is made, and if the message processing service instance and the sub-table instance are not found, the next step is carried out;
2) the subscription id configurator finds the name of the corresponding message processing service, the maximum number of the message processing services, the stored sub-table name and the stored maximum number of the sub-tables in the subscription id type configuration according to the type of the subscription id;
3) according to the message processing service name, finding a corresponding latest message processing service instance in the message processing service instance configuration; if the number of the latest message processing service instances is larger than or equal to the configuration number, adding a new message processing service instance, if the number of the latest message processing service instances is smaller than the configuration number, returning to the current message processing service instance, and adding 1 to the number of the current message processing service instances; if the corresponding latest message processing service instance cannot be found, a new message processing service instance is created.
4) According to the message processing service instance, finding a corresponding latest sub-table instance in the message processing service sub-table instance configuration; if the processing number of the sub-table instances is larger than or equal to the configured number, adding one sub-table instance, if the processing number of the sub-table instances is smaller than the configured number, returning to the current sub-table instance, adding 1 to the processing number of the current sub-table instance, and if the sub-table instances cannot be found, creating a new sub-table instance.
5) And the main program returns the corresponding message processing service instance and the sub-table instance of the subscription id, and a record is newly added in the configuration of the mapping position of the subscription id.
Example 1:
in the embodiment, 1 message logic service is realized by a spring boot, 2 message processing services are realized by the spring boot, and 2 mysql database instances comprise 4 message subscription configuration tables and 4 message storage sub-tables. The message logic service is responsible for overall scheduling of the program, including inquiring or newly adding subscription id configuration, and calling the message processing service to store and synchronize the subscription message.
As shown in fig. 2, after the message logic service receives a message storage request, a subscription id configurator (PartitionService) is called by a message subscriber (subscribe service) to query a processing service instance and a storage sub-table corresponding to a subscription id. And finally, calling an im-partition-description-xxx (im-partition-descriptor-xxx) to store the message.
As shown in fig. 3, after the message logic service receives the message synchronization request, the message subscriber (subscription service) first calls a subscription id configurator (PartitionService) to query the processing service instance and the storage sub-table corresponding to the subscription id. And finally, calling an im-partition-description-xxx message processing service instance to perform message query.
The invention can distinguish single chat situations and group chat situations, designs the subscription id, uniformly uses the subscription id to synchronize or store messages by a program, uses the number of group chat users as a weight to determine whether to use a read diffusion mode or a write diffusion mode to store the messages of the users, uses a multi-thread query mode to reduce the time of message synchronization, and designs a set of mechanism on the basis to enable the messages of different subscription ids to be processed by different services in a database-sharing and table-sharing mode.
The above description is directed to the preferred embodiments of the present invention, but the embodiments are not intended to limit the scope of the claims of the present invention, and all equivalent changes and modifications made within the technical spirit of the present invention should fall within the scope of the claims of the present invention.
Claims (6)
1. A method for storing and synchronizing messages in an instant messaging system is characterized by comprising the following steps:
s1, when the main program of the message receives the request of message storage, the main program calls the subscription object creating interface of the message subscriber, the subscriber generates a subscription message object according to the message type, and the subscription message object contains subscription id, diffusion type and message content;
s2, calling a subscription id configurator by the main program according to the subscription id of the subscription message object to acquire a message processing service address and a sub-table name corresponding to the subscription id, and returning the message processing service address and the sub-table name by the subscription id configurator according to the rule;
s3, the main program calls the message processing service according to the message service address to store the subscription message object into the corresponding sub-table;
s4, when the main program receives the message synchronization request, the main program calls a message subscriber inquiry interface, the inquiry interface inquires the subscription id array of the user, the subscription id array is divided into 4 parts, and 4 threads are used for inquiring messages corresponding to the 4 subscription ids;
s5, traversing the subscription id array by the main program, and calling a subscription id configurator to acquire the message processing service and the sub-table corresponding to the subscription id;
and S6, calling the message processing service by the main program to inquire the message corresponding to the subscription id from the corresponding sub-table, and merging all the subscription messages and returning after the 4 threads are processed.
2. The method for storing and synchronizing instant messaging system messages according to claim 1, wherein in the step S1, the message types include single chat information and group chat information, and a user id of sending the message and a user id of receiving the message are used as subscription ids in case of the single chat information, and a group id is used as subscription ids in case of the group chat message.
3. The method for storing and synchronizing instant messaging system messages according to claim 1, wherein the subscription id configurator structure comprises:
subscription id type configuration, which is used for configuring the names of message processing services with different subscription id types, the maximum number of the message processing services, the stored sub-table names and the maximum number of the stored sub-tables;
the message processing service instance configuration is used for recording the total number of the subscription ids corresponding to each message processing service currently;
the message processing service sub-table instance configuration is used for recording the total number of the subscription ids currently corresponding to each sub-table in each message processing service; and
and the subscription id maps the position configuration, and the position configuration is used for recording the message processing service instance and the sub-table instance corresponding to the subscription id.
4. The method for storing and synchronizing instant messaging system messages according to claim 1, wherein the subscription id mapping location query and generation comprises the steps of:
1) the main program inquires a corresponding message processing service instance and a sub-table instance in the configuration of the subscription id mapping position according to the subscription id, if the message processing service instance and the sub-table instance are found, the return is made, and if the message processing service instance and the sub-table instance are not found, the next step is carried out;
2) the subscription id configurator finds the name of the corresponding message processing service, the maximum number of the message processing services, the stored sub-table name and the stored maximum number of the sub-tables in the subscription id type configuration according to the type of the subscription id;
3) according to the message processing service name, finding a corresponding latest message processing service instance in the message processing service instance configuration;
4) according to the message processing service instance, finding a corresponding latest sub-table instance in the message processing service sub-table instance configuration;
5) and the main program returns the corresponding message processing service instance and the sub-table instance of the subscription id, and a record is newly added in the configuration of the mapping position of the subscription id.
5. The method for storing and synchronizing messages in an instant messaging system according to claim 4, wherein in the step 3), if the number of the latest message processing service instances processed is greater than or equal to the configuration number, a new message processing service instance is added, and if the number of the latest message processing service instances processed is less than the configuration number, the current message processing service instance is returned, and the number of the current message processing service instances processed is increased by 1; if the corresponding latest message processing service instance cannot be found, a new message processing service instance is created.
6. The method for storing and synchronizing messages in an instant messaging system according to claim 4, wherein in the step 4), if the processing number of the sub-table instances is greater than or equal to the configured number, a new sub-table instance is added, if the processing number is less than the configured number, the current sub-table instance is returned, and the processing number of the current sub-table instance is added by 1, and if the sub-table instance cannot be found, a new sub-table instance is created.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010476297.4A CN111669313B (en) | 2020-05-29 | 2020-05-29 | Method for storing and synchronizing instant communication system message |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010476297.4A CN111669313B (en) | 2020-05-29 | 2020-05-29 | Method for storing and synchronizing instant communication system message |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111669313A CN111669313A (en) | 2020-09-15 |
| CN111669313B true CN111669313B (en) | 2022-05-03 |
Family
ID=72385242
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010476297.4A Active CN111669313B (en) | 2020-05-29 | 2020-05-29 | Method for storing and synchronizing instant communication system message |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111669313B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685537A (en) * | 2020-12-28 | 2021-04-20 | 北京达佳互联信息技术有限公司 | Session record query method, device and related equipment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030212818A1 (en) * | 2002-05-08 | 2003-11-13 | Johannes Klein | Content based message dispatch |
| CN1783849A (en) * | 2004-12-03 | 2006-06-07 | 腾讯科技（深圳）有限公司 | News subscribing method and system based on immediate communication platform |
| US8489690B2 (en) * | 2008-08-28 | 2013-07-16 | International Business Machines Corporation | Providing cellular telephone subscription for e-mail threads |
| EP2869578A1 (en) * | 2013-11-01 | 2015-05-06 | Nagravision S.A. | Method and apparatus for distributing media licenses within a distribution system of a secure multimedia service |
| CN104408132B (en) * | 2014-11-28 | 2018-03-02 | 北京京东尚科信息技术有限公司 | Data push method and system |
| CN108737246B (en) * | 2018-05-07 | 2021-02-02 | 北京达佳互联信息技术有限公司 | Message issuing method, device, storage medium and server |
| CN109818851A (en) * | 2019-01-23 | 2019-05-28 | 北京奇艺世纪科技有限公司 | A kind of information persistence method, device and electronic equipment |
| CN111083037B (en) * | 2019-10-22 | 2022-02-01 | 贝壳技术有限公司 | Method, device, medium and electronic equipment for realizing instant messaging |
| CN111177363B (en) * | 2019-12-30 | 2023-07-11 | 广州市百果园网络科技有限公司 | Writing method and device of group message, server and storage medium |
-
2020
- 2020-05-29 CN CN202010476297.4A patent/CN111669313B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111669313A (en) | 2020-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107395697B (en) | Push channel selection method, message push method, device and equipment, and readable medium | |
| CN106598705B (en) | Asynchronous task scheduling method, device and system and electronic equipment | |
| US10915554B2 (en) | Database replication system | |
| CN108804234B (en) | Data storage system and method of operation thereof | |
| CN101236569B (en) | Highly effective dynamic path analysis method based on ContextFS | |
| CN100437590C (en) | Method for prefetching object | |
| CN112181306A (en) | Method for generating distributed ID (identity) in number segment mode | |
| EP3809676A1 (en) | Dns resolution method, authoritative dns server and dns resolution system | |
| CN105808776A (en) | Data management system and method of distributed database | |
| CN105991660B (en) | System for resource sharing among multiple cloud storage systems | |
| CN111669313B (en) | Method for storing and synchronizing instant communication system message | |
| CN106549857A (en) | A kind of method and system of trans-regional PUSH message | |
| CN107315745B (en) | Private letter storage method and system | |
| CN101102286A (en) | A method and device for referring to user information in instant communication system | |
| JP2002251305A (en) | Method for using memory area in terminal | |
| CN109145053B (en) | Data processing method and device, client and server | |
| CN104539583A (en) | Real-time database subscription system and method | |
| CN112003922A (en) | Data transmission method and device | |
| CN114063936B (en) | Method, system, equipment and storage medium for optimizing timing task | |
| CN108399175B (en) | Data storage and query method and device | |
| CN113407560B (en) | Update message processing method, data synchronization method and configuration information configuration method | |
| CN116069489A (en) | Automatic load balancing distribution method and system | |
| CN105813037B (en) | Short message concurrent service processing method and device | |
| CN105868210A (en) | Creating method and device of unique index in distributed database | |
| CN105162684A (en) | Group communication system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |