CN114490026A - Message consumption optimization method and terminal - Google Patents
Message consumption optimization method and terminal Download PDFInfo
- Publication number
- CN114490026A CN114490026A CN202111570368.8A CN202111570368A CN114490026A CN 114490026 A CN114490026 A CN 114490026A CN 202111570368 A CN202111570368 A CN 202111570368A CN 114490026 A CN114490026 A CN 114490026A
- Authority
- CN
- China
- Prior art keywords
- messages
- consumption
- message
- service
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
Abstract
The invention discloses a message consumption optimization method and a terminal; establishing consumption main threads with corresponding quantity according to the quantity of storage areas of the kafka system, and establishing preset quantity of sub-threads by each consumption main thread; respectively acquiring a preset number of messages from each partition of the kafka system by a consumption main thread, and grouping the messages according to a first service main key of the messages and a preset service logic condition; the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing; the consumption main thread is set for each storage on the kafka, the design mode of the kafka is skipped, the main thread consumes the messages in the partition, the messages are grouped, and the sub-threads process the messages according to the grouping result, so that the consumption throughput of the kafka is improved, the consumption rate of the messages can be controlled and adjusted by adjusting the number of the threads, and the kafka storage system is more flexible and changeable.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a message consumption optimization method and a terminal.
Background
The purpose of most MQ projects is to decouple different modules, so that a large amount of data can be stacked in a memory or file form, and the process of stacking messages is performed in an asynchronous mode. Kafka, among others, is a common high-throughput distributed publish-subscribe messaging system that can handle all the action flow data in a consumer-scale web site. The number of consuming threads in the Kafka architecture has a relationship with the partition (storage area) of Topic (Topic is a transmission medium between a message publisher (Pub) and a subscriber (Sub)) (one type of message has one Topic, and the partition number of the Topic needs to be fixed when each Topic is created), and at a certain time, only one consumer thread can be connected to one partition. Problems may arise in this case: when the amount of messages that a project needs to process reaches a certain bottleneck, the consumption rate of the messages is limited.
In view of the above problems, the existing systems generally increase the number of partitions of the topic to increase the number of implementation of the consumers, but this method needs to delete the topic to rebuild the number of partitions related to the topic every time the number of consumers needs to be increased, or to set a larger number of partitions when the topic is newly built for the first time, which is inflexible and the number of partitions can not be expanded infinitely. And flexible expansion cannot be achieved when the scene of the messages needing to be consumed in sequence is targeted.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the message consumption optimization method and the terminal are provided, and the message consumption rate can be flexibly controlled.
In order to solve the technical problems, the invention adopts the technical scheme that:
a method for optimizing message consumption, comprising the steps of:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
a message consumption optimizing terminal comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
The invention has the beneficial effects that: the technical scheme of the invention sets a consumption main thread aiming at each storage on kafka, and breaks away from the design mode of the kafka, the main thread consumes a message in a partition, meanwhile, the message is grouped through a first service main key and a preset service logic condition, and the messages are processed by a plurality of sub-threads according to the grouping result, so that the consumption rate of the message can be greatly improved, and the consumption throughput of the kafka is improved; and for the control of the consumption rate, the partition does not need to be modified, the consumption rate of the message can be controlled and adjusted by adjusting the number of threads, and the method is more flexible and changeable.
Drawings
FIG. 1 is a flow chart of a method for optimizing message consumption according to an embodiment of the present invention;
fig. 2 is a structural diagram of a message consumption optimizing terminal according to an embodiment of the present invention;
description of reference numerals:
1. an optimized terminal for message consumption; 2. a processor; 3. a memory.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
Referring to fig. 1, a method for optimizing message consumption includes the steps of:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
From the above description, the beneficial effects of the present invention are: the technical scheme of the invention sets a consumption main thread aiming at each storage on kafka, and breaks away from the design mode of the kafka, the main thread consumes a message in a partition, meanwhile, the message is grouped through a first service main key and a preset service logic condition, and the messages are processed by a plurality of sub-threads according to the grouping result, so that the consumption rate of the message can be greatly improved, and the consumption throughput of the kafka is improved; and for the control of the consumption rate, the partition does not need to be modified, the consumption rate of the message can be controlled and adjusted by adjusting the number of threads, and the method is more flexible and changeable.
Further, the grouping the messages according to the first service primary key of the message and the preset service logic condition in step S2 includes:
identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set;
the step S3 specifically includes:
and randomly distributing all the messages recorded in the non-repeating set to self sub-threads by the consumption main thread for concurrent processing, simultaneously randomly distributing the messages recorded in the repeating set to self sub-threads by taking a group as a unit, and sequentially processing the messages by the sub-threads according to the time information in the messages.
As can be seen from the above description, when there are a plurality of messages having a certain service key and the plurality of messages have a time sequence, the messages are divided into a group and processed by the same sub-thread, so that sequential processing can be performed according to time information, and the problem of processing sequence disorder caused by allocating the messages to a plurality of threads is avoided.
Further, the identifying a first service primary key in all the messages, grouping two or more messages with the same first service primary key into a group and recording the group into a duplicate set, and recording the rest messages into a non-duplicate set specifically includes:
identifying a first service primary key in all the messages, calculating a hash value of the first service primary key, judging whether the messages have the same first service primary key according to the hash value, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set.
According to the above description, whether the first service main keys are the same or not is judged by calculating the hash value of the first service message main key and comparing the hash value with the hash value, so that the comparison is more accurate.
Further, the sequentially processing the message by the child thread according to the time information in the message specifically includes:
and judging whether the number of the messages belonging to the same group is larger than a preset threshold value or not by the sub-thread, if so, grouping the messages again according to a second service main key to obtain secondary grouping information, and creating corresponding grandchild threads according to the secondary grouping information to process the messages.
From the above description, when the message data volume of a packet is large (larger than the preset threshold), we continue to perform secondary packet on the message, and create a grandchild thread corresponding to the packet number for processing, and improve the efficiency of message processing by expanding the thread number of message processing.
Further, the step S3 is followed by the step of:
s31, judging whether all the messages are processed completely, if so, returning to the step S2;
and S4, receiving a user request, and adjusting the number of the sub-threads of each consumption main thread according to the user request.
From the above description, after the current batch of messages is processed, the batch of messages is pulled to kafka for processing, and meanwhile, the user can control the message processing efficiency by adjusting the number of sub-threads.
Referring to fig. 2, a message consumption optimizing terminal includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
From the above description, the beneficial effects of the present invention are: the technical scheme of the invention sets a consumption main thread aiming at each storage on kafka, and breaks away from the design mode of the kafka, the main thread consumes a message in a partition, meanwhile, the message is grouped through a first service main key and a preset service logic condition, and the messages are processed by a plurality of sub-threads according to the grouping result, so that the consumption rate of the message can be greatly improved, and the consumption throughput of the kafka is improved; and for the control of the consumption rate, the partition does not need to be modified, the consumption rate of the message can be controlled and adjusted by adjusting the number of threads, and the method is more flexible and changeable.
Further, the grouping the messages according to the first service primary key of the message and the preset service logic condition in step S2 includes:
identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set;
the step S3 specifically includes:
and randomly distributing all the messages recorded in the non-repeating set to self sub-threads by the consumption main thread for concurrent processing, simultaneously randomly distributing the messages recorded in the repeating set to self sub-threads by taking a group as a unit, and sequentially processing the messages by the sub-threads according to the time information in the messages.
As can be seen from the above description, when there are a plurality of messages having a certain service key and the plurality of messages have a time sequence, the messages are divided into a group and processed by the same sub-thread, so that sequential processing can be performed according to time information, and the problem of processing sequence disorder caused by allocating the messages to a plurality of threads is avoided.
Further, the identifying a first service primary key in all the messages, grouping two or more messages with the same first service primary key into a group and recording the group into a duplicate set, and recording the rest messages into a non-duplicate set specifically includes:
identifying a first service primary key in all the messages, calculating a hash value of the first service primary key, judging whether the messages have the same first service primary key according to the hash value, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set.
According to the above description, whether the first service main keys are the same or not is judged by calculating the hash value of the first service message main key and comparing the hash value with the hash value, so that the comparison is more accurate.
Further, the sequentially processing the message by the child thread according to the time information in the message specifically includes:
and judging whether the number of the messages belonging to the same group is larger than a preset threshold value or not by the sub-thread, if so, grouping the messages again according to a second service main key to obtain secondary grouping information, and creating corresponding grandchild threads according to the secondary grouping information to process the messages.
From the above description, when the message data volume of a packet is large (larger than the preset threshold), we continue to perform secondary packet on the message, and create a grandchild thread corresponding to the packet number for processing, and improve the efficiency of message processing by expanding the thread number of message processing.
Further, the step S3 is followed by the step of:
s31, judging whether all the messages are processed completely, if so, returning to the step S2;
and S4, receiving a user request, and adjusting the number of the sub-threads of each consumption main thread according to the user request.
From the above description, after the current batch of messages is processed, the batch of messages is pulled to kafka for processing, and meanwhile, the user can control the message processing efficiency by adjusting the number of sub-threads.
The message consumption optimization method and the terminal are applied to MQ (message queue) projects based on a kafka architecture for processing messages.
Referring to fig. 1, a first embodiment of the present invention is:
a method for optimizing message consumption, comprising the steps of:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
in the embodiment, a relevant kafka cluster relevant environment is set up in advance, a relevant tool jar package is downloaded, and relevant kafka consumer client codes are written. An associated consuming thread is created by the consumer client program.
S2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
in step S2, the grouping the message according to the first service key of the message and the preset service logic condition includes:
identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set;
the identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set specifically comprises:
identifying a first service primary key in all the messages, calculating a hash value of the first service primary key, judging whether the messages have the same first service primary key according to the hash value, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set.
In this embodiment, the program pulls a batch of messages from kafka through method call, and the size and number of the pulled messages may be preset. There are 5 partitions in kafka in this embodiment, then there are 5 consuming main threads pulling messages to these 5 partitions.
For the acquired batch of messages, grouping is carried out through certain service logic conditions. Specifically, hash processing is performed according to a key in the message, that is, the first service key, the key of the message is compared based on the hash value, and grouping is performed according to a comparison result. Say 100, each message has at least one primary service key, which may be repeated to represent multiple messages of a record. For example, if all the messages have userId fields, the messages with different userId represent different account messages, and if the userId is consistent, the messages represent multiple messages of the same account, and at this time, the messages need to be sequentially consumed. We perform deduplication based on userId, which is recorded into a non-duplicate set when there are only 1 record for each userId. When there are at least 2 messages for userId, then group and record into duplicate sets.
S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing;
the step S3 specifically includes:
randomly distributing all the messages recorded in the non-repeating set to self sub-threads by the consumption main thread for concurrent processing, simultaneously randomly distributing the messages recorded in the repeating set to self sub-threads by taking a group as a unit, and sequentially processing the messages by the sub-threads according to the time information in the messages;
the sub-thread sequentially processes the message according to the time information in the message specifically comprises:
and judging whether the number of the messages belonging to the same group is larger than a preset threshold value or not by the sub-thread, if so, grouping the messages again according to a second service main key to obtain secondary grouping information, and creating corresponding grandchild threads according to the secondary grouping information to process the messages.
In this embodiment, the data in the non-duplicate set may be directly and evenly distributed to a plurality of subtask threads for concurrent consumption, because there is no problem in order, while the data in the duplicate set is distributed in units of each group. Meanwhile, when the message volume of a certain service primary key continues to increase, the subtask processing thread needs to be supported to continue to perform hash comparison according to a second service primary key and then to perform assignment again, and then to distribute the message to a grandchild task processing thread for subsequent processing. For example, after the amount of synchronization data of a certain userId account reaches a preset threshold, we may group the messages according to another key of the message, that is, a second service primary key, for example, according to the target location of the message, group the messages again, and allocate the messages to a grandchild thread according to the group, and the grandchild thread sequentially processes the messages according to the group. In addition, for some specific first service primary key or second service primary key, a special thread can be configured in advance to be specially responsible for processing or increase the number of sub-threads for processing.
The step S3 is followed by the step of:
s31, judging whether all the messages are processed completely, if yes, returning to the step S2.
And S4, receiving a user request, and adjusting the number of the sub-threads of each consumption main thread according to the user request.
In this embodiment, when the number of messages needs to be consumed in a large amount, the number of sub-threads may be artificially increased to increase the consumption throughput, so that the consumption rate of the flexible and variable control messages may be realized, and the throughput of message processing may be increased or decreased.
Referring to fig. 2, the second embodiment of the present invention is:
a message consumption optimizing terminal comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the first embodiment when executing the computer program.
The main principle of the invention is that a consumer thread processes a partition message, and acquires a batch of messages from the partition message and distributes the batch of messages to different sub-threads for processing according to a service main key, namely, the design mode of kafka is skipped, and the messages are processed in a multi-thread mode, so that the consumption rate of sequential messages is greatly improved, and the consumption throughput of kafka is improved.
In summary, according to the optimization method and terminal for message consumption provided by the invention, a consumption main thread is set for each storage on kafka, the design mode of kafka is skipped, a message in a partition is consumed by one main thread, meanwhile, the message is grouped through a first service main key and preset service logic conditions, and the message is processed by a plurality of sub-threads according to the grouping result, so that the consumption rate of the message can be greatly improved, and the consumption throughput of the kafka is improved; in addition, for the control of the consumption rate, the partition does not need to be modified, the consumption rate of the message can be controlled and adjusted by adjusting the number of threads, and the method is more flexible and changeable; and meanwhile, the messages are grouped according to the service master key, and the messages in the same group are processed by the same thread machine type, so that the order of processing the messages is not disturbed.
The above description is only an embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent modifications made by the present invention and the contents of the accompanying drawings, which are directly or indirectly applied to the related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for optimizing message consumption, comprising the steps of:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
2. The method of claim 1, wherein the grouping the messages according to the first service primary key of the message and the preset service logic condition in step S2 includes:
identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set;
the step S3 specifically includes:
and randomly distributing all the messages recorded in the non-repeating set to self sub-threads by the consumption main thread for concurrent processing, simultaneously randomly distributing the messages recorded in the repeating set to self sub-threads by taking a group as a unit, and sequentially processing the messages by the sub-threads according to the time information in the messages.
3. The method according to claim 2, wherein the identifying a first service primary key in all the messages, grouping and recording two or more messages with the same first service primary key into a duplicate set, and recording the rest of the messages into a non-duplicate set specifically comprises:
identifying a first service primary key in all the messages, calculating a hash value of the first service primary key, judging whether the messages have the same first service primary key according to the hash value, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set.
4. The method according to claim 2, wherein the sub-thread sequentially processes the message according to the time information in the message specifically comprises:
and judging whether the number of the messages belonging to the same group is larger than a preset threshold value or not by the sub-thread, if so, grouping the messages again according to a second service main key to obtain secondary grouping information, and creating corresponding grandchild threads according to the secondary grouping information to process the messages.
5. The message consumption optimizing method of claim 1, wherein the step S3 is further followed by the steps of:
s31, judging whether all the messages are processed completely, if so, returning to the step S2;
and S4, receiving a user request, and adjusting the number of the sub-threads of each consumption main thread according to the user request.
6. A message consumption optimizing terminal comprising a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of:
s1, establishing a corresponding number of consumption main threads according to the number of the storage areas of the kafka system, and establishing a preset number of sub-threads by each consumption main thread;
s2, the consumption main thread respectively acquires a preset number of messages from each partition of the kafka system, and the messages are grouped according to a first service main key of the messages and a preset service logic condition;
and S3, the consumption main thread distributes the message to the sub-thread of the consumption main thread according to the grouping result for processing.
7. The terminal of claim 6, wherein the grouping the messages according to the first service primary key of the message and the preset service logic condition in step S2 includes:
identifying a first service primary key in all the messages, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set;
the step S3 specifically includes:
and randomly distributing all the messages recorded in the non-repeating set to self sub-threads by the consumption main thread for concurrent processing, simultaneously randomly distributing the messages recorded in the repeating set to self sub-threads by taking a group as a unit, and sequentially processing the messages by the sub-threads according to the time information in the messages.
8. The terminal of claim 7, wherein the identifying a first service primary key in all the messages, grouping and recording two or more messages with the same first service primary key into a duplicate set, and recording the rest of the messages into a non-duplicate set specifically comprises:
identifying a first service primary key in all the messages, calculating a hash value of the first service primary key, judging whether the messages have the same first service primary key according to the hash value, grouping more than two messages with the same first service primary key into a group and recording the group into a repeated set, and recording the rest messages into a non-repeated set.
9. The terminal of claim 7, wherein the sub-thread sequentially processes the message according to the time information in the message specifically comprises:
and judging whether the number of the messages belonging to the same group is larger than a preset threshold value or not by the sub-thread, if so, grouping the messages again according to a second service main key to obtain secondary grouping information, and creating corresponding grandchild threads according to the secondary grouping information to process the messages.
10. The message consumption optimizing terminal of claim 6, wherein the step S3 is further followed by the steps of:
s31, judging whether all the messages are processed completely, if so, returning to the step S2;
and S4, receiving a user request, and adjusting the number of the sub-threads of each consumption main thread according to the user request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111570368.8A CN114490026A (en) | 2021-12-21 | 2021-12-21 | Message consumption optimization method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111570368.8A CN114490026A (en) | 2021-12-21 | 2021-12-21 | Message consumption optimization method and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114490026A true CN114490026A (en) | 2022-05-13 |
Family
ID=81494912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111570368.8A Pending CN114490026A (en) | 2021-12-21 | 2021-12-21 | Message consumption optimization method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114490026A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115866017A (en) * | 2023-02-27 | 2023-03-28 | 天翼云科技有限公司 | Message processing method, message processing device, communication equipment and storage medium |
-
2021
- 2021-12-21 CN CN202111570368.8A patent/CN114490026A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115866017A (en) * | 2023-02-27 | 2023-03-28 | 天翼云科技有限公司 | Message processing method, message processing device, communication equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112099958B (en) | Distributed multi-task management method and device, computer equipment and storage medium | |
| CN103412786A (en) | High performance server architecture system and data processing method thereof | |
| CN107818012B (en) | Data processing method and device and electronic equipment | |
| CN107908471B (en) | Task parallel processing method and processing system | |
| CN110648178A (en) | Method for increasing kafka consumption capacity | |
| CN113452774B (en) | Message pushing method, device, equipment and storage medium | |
| US20230244537A1 (en) | Efficient gpu resource allocation optimization method and system | |
| CN108304236B (en) | User interface refreshing method based on message subscription under cloud platform | |
| CN114827049B (en) | Pile-up data consumption method based on kafka, terminal equipment and storage medium | |
| CN114490026A (en) | Message consumption optimization method and terminal | |
| CN104144202A (en) | Hadoop distributed file system access method, system and device | |
| CN109902059A (en) | A kind of data transmission method between CPU and GPU | |
| CN108595121B (en) | Data storage method and device | |
| CN109388501B (en) | Communication matching method, device, equipment and medium based on face recognition request | |
| CN102970349B (en) | A kind of memory load equalization methods of DHT network | |
| US20100180280A1 (en) | System and method for batch resource allocation | |
| CN112099930A (en) | Quantum computer cluster distributed queue scheduling method | |
| CN109005465B (en) | Bullet screen message distribution method, device, equipment and storage medium | |
| CN107911484B (en) | Message processing method and device | |
| CN113918333A (en) | Task processing method and terminal | |
| CN115695446A (en) | Openstack-based high-concurrency mirror image transmission method | |
| CN112395063B (en) | Dynamic multithreading scheduling method and system | |
| CN114860449A (en) | Data processing method, device, equipment and storage medium | |
| CN114598662A (en) | Message queue cluster federal management system and method | |
| CN113342886A (en) | Data exchange method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |