CN112685191B - Message processing method and device - Google Patents
Message processing method and device Download PDFInfo
- Publication number
- CN112685191B CN112685191B CN201910992376.8A CN201910992376A CN112685191B CN 112685191 B CN112685191 B CN 112685191B CN 201910992376 A CN201910992376 A CN 201910992376A CN 112685191 B CN112685191 B CN 112685191B
- Authority
- CN
- China
- Prior art keywords
- data
- data message
- target data
- message
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data message processing method and a device based on a plurality of business lines, which are applied to a data processing system, wherein the data message of each business line comprises a plurality of fields in a target data table; the method comprises the following steps: pre-creating a plurality of consumption processes; receiving data messages from each service stripe; determining target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message; and determining a consumption process corresponding to the target data, and sending the received data message to the consumption process corresponding to the target data, so that the consumption process writes the values of a plurality of fields included in the received data message into the target data table. The application can solve the problems of dirty data reading, data version conflict and data loss caused by the out-of-order consumption of the messages under high concurrency.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing a message.
Background
In the data processing process, due to the diversification of services, data sources of different service lines need to be written into the same ES type (namely, ES wide table), and each data source enters the data processing system as a kafka message to be consumed/processed by corresponding service processing logic. However, since the sequence of the kafka message reaching the data processing system of each data source cannot be guaranteed, when a certain data source has no primary key, double writing is needed to similarly guarantee the sequential consumption of the messages, but under the condition of high concurrency, the method may cause the problems of dirty data reading, ES version conflict, data loss and the like.
Disclosure of Invention
In view of the above, the present application aims to provide a message processing method and apparatus, which can solve the problems of dirty data reading, data version conflict and data loss caused by out-of-order consumption of messages under high concurrency.
In order to achieve the above purpose, the present application provides the following technical solutions:
a data message processing method based on a plurality of business lines is applied to a data processing system, and the data message of each business line comprises a plurality of fields in a target data table; the method comprises the following steps:
pre-creating a plurality of consumption processes;
receiving data messages from each service stripe;
determining target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message;
and determining a consumption process corresponding to the target data, and sending the received data message to the consumption process corresponding to the target data, so that the consumption process writes the values of a plurality of fields included in the received data message into the target data table.
A data message processing device based on a plurality of business lines is applied to a data processing system, and the data message of each business line comprises a plurality of fields in a target data table; the device comprises:
a creation unit for creating a plurality of consumption processes in advance;
a receiving unit for receiving data messages from each service line;
a determining unit, configured to determine target data in a target data table corresponding to the data message according to values of a plurality of fields included in the received data message;
and the hash unit is used for determining a consumption process corresponding to the target data, sending the received data message to the consumption process corresponding to the target data, and enabling the target consumption process to write the values of a plurality of fields included in the received data message into the target data table.
An electronic device, comprising: at least one processor, and a memory connected to the at least one processor via a bus; the memory stores one or more computer programs executable by the at least one processor; the at least one processor, when executing the one or more computer programs, implements the steps in the data message processing method based on the plurality of business strip lines.
A computer readable storage medium storing one or more computer programs which when executed by a processor implement the data message processing method based on a plurality of traffic stripes described above.
According to the technical scheme, a plurality of consumption processes are created in advance, and for the data messages from different business lines, the messages are classified into different consumption processes according to the target data in the target data table corresponding to the data messages, so that the data messages corresponding to the same target data are sent into the same consumption process for processing, and the problems of dirty data reading, data version conflict and data loss caused by out-of-order consumption of the high-concurrency message can be solved.
Drawings
FIG. 1 is a schematic diagram of a prior art data processing system double writing process for a waybill message and a shipping message;
FIG. 2 is a flow chart of a method for processing a data message based on a plurality of business striping in accordance with an embodiment of the present application;
FIG. 3 is a schematic diagram of a data message processing apparatus based on multiple service stripes according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a schematic diagram of a two-write process of a data processing system to a manifest message (including a manifest number and a manifest status) and a shipping message (including a package number, a site id, a manifest number):
1) When the waybill message is consumed, the waybill state (i.e. waybill_state in fig. 1) is written into the redis, meanwhile, a main key (i.e. pk_code|siteid in fig. 1) is obtained from the redis of the shipping message through the step 1, and then the waybill state is written into the ES based on the obtained main key through the step 3;
2) When the shipping information is consumed, the main key is written into the redis, meanwhile, the waybill state is acquired from the redis of the waybill information through the step 2, and then the fields such as the main key and the like in the shipping information and the acquired waybill state are written into the ES wide table through the step 4.
In the process shown in fig. 1, in the process of consuming shipping information under high concurrency, after acquiring the waybill state, writing the fields such as the primary key in the shipping information and the acquired waybill state into the ES through step 4, receiving a new waybill message again, and writing the new waybill state in the new waybill message into the ES through steps 1 and 3, and then executing step 4 to finish, wherein the new waybill state is covered by the old waybill state written when passing through step 4, so that the waybill state in the ES is not the latest value. In addition, in the case of high concurrency, there is a possibility that step 3 and step 4 write data of the same primary key to the ES at the same time, resulting in ES version conflict and lost data problems.
As can be seen from the example given in fig. 1, in high concurrency situations, message out-of-order consumption can cause data dirty reads, data version conflicts, and data loss problems.
In order to solve the above-mentioned problem, in the embodiment of the present application, a plurality of consuming processes are used to process data messages from a plurality of service lines, where the data message of each service line includes a plurality of fields in a target data table. By hashing data messages from different service lines but corresponding to the same primary key in the target data table (each piece of data in the target data table, having a primary key value uniquely identifying the piece of data) into the same consumption process, even under the condition of high concurrency, the condition of message disorder is avoided, the condition of simultaneously writing data of the same primary key into the target data is avoided, and therefore, even under the condition of high concurrency, the condition of message disorder consumption is avoided, and therefore, the problems of dirty data reading, data version conflict, data loss and the like are avoided.
Referring to fig. 2, fig. 2 is a flowchart of a method for processing a data message based on a plurality of service lines according to an embodiment of the present application, where the method is applied to a data processing system, and a data message of each service line includes a plurality of fields in a target data table; as shown in fig. 2, the method comprises the steps of:
step 201, a plurality of consumption processes are created in advance;
in the embodiment of the application, the number of the created consumption processes can be preset.
Step 202, receiving data messages from each service stripe.
In the embodiment of the application, the data messages of different service lines are concurrent, a plurality of data fields included in the data messages of the service lines belong to fields in the same target data table, and are directly associated or indirectly associated according to associated fields included in the data messages.
For example, in the case that the manifest message includes a manifest number and a manifest status, the primary key and the manifest number are included in the shipping message. Because both data messages have the waybill number field, the waybill number field is used as an association field to directly associate the two data messages, and if the value of the waybill number field in the waybill message is the same as the value of the waybill number field in the shipping message, it may be determined that both data messages correspond to the same data in the target data table (here, the same data refers to one piece of data in the target data table).
Step 203, determining target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message.
In the embodiment of the application, each piece of data in the target data table comprises a primary key value which uniquely identifies the piece of data. Among the plurality of fields included in the data message of at least one service bar of the plurality of service bar data messages, a pre-designated primary key field (the primary key field may be a combination of a plurality of fields) exists.
For example, the target data table includes therein a field A, B, C, D, in which a field a is designated in advance as a primary key field, capable of uniquely identifying one piece of data in the target data table. The data message of the service strip line 1 comprises a field A and a field B, the data message of the service strip line 2 comprises a field B and a field C, and the data message of the service strip line 3 comprises a field C and a field D. The data message of the service strip line a includes a primary key field, i.e., a field a.
In the embodiment of the present application, there are associated fields in a plurality of fields included in a data message of each service strip, and data messages in different service strips implement direct association or indirect association through the same associated field, where data messages in different service strips implement indirect association through the same associated field, and the method includes:
the data messages in the two business lines respectively have the same associated fields with the data messages in the other business line, so that the first indirect association of the data messages in the two business lines is realized;
or alternatively, the process may be performed,
the data messages in the two business lines are respectively and indirectly related to the data messages in the other business line;
or alternatively, the process may be performed,
the data messages in two traffic lanes are directly associated and first indirectly associated, respectively, with the data messages in the other traffic lane. .
For example, the data message of the service stripe 1 and the data message of the service stripe 2 are directly associated because they both include the field B, the data message of the service stripe 2 and the data message of the service stripe 3 are directly associated because they both include the field C, and the data message of the service stripe 1 and the data message of the service stripe 3 are directly associated and are indirectly associated for the first time because they both are directly associated with the data message of the service stripe 2.
In the embodiment of the present application, the determining, according to the values of the plurality of fields included in the received data message, the target data in the target data table corresponding to the data message includes:
if a main key field exists in a plurality of fields included in the data message, determining target data in a target data table corresponding to the received data message according to the value of the main key field;
otherwise, the data message of any other business strip line which has direct association or indirect association relation with the received data message and comprises a main key field is found out through the value of the association field in the received data message, and the target data in the target data table corresponding to the received data message is determined according to the value of the main key field of any other business strip line.
For example, assume that the following data message is currently received:
the data messages of the 2 service lines 1 are assumed to be data messages 1 and data messages 2, wherein the values of the fields A and B in the data messages 1 are x1 and y1 respectively, and the values of the fields A and B in the data messages 2 are x2 and y2 respectively;
the data message of 1 service strip line 2 is assumed to be data message 3, and the values of fields B and C in the data message 3 are y1 and z1;
the data messages of the 2 service stripes 3 are assumed to be data message 4 and data message 5, wherein the values of the fields C and D in the data message 4 are z1 and q1 respectively, and the values of the fields C and D in the data message 5 are z2 and q2 respectively.
Then the first time period of the first time period,
it can be determined that the data message 1, the data message 3 and the data message 4 all correspond to target data with a primary key value of x1 in the target data table; for the data message 1, the target data corresponding to the data message 1 can be determined directly according to the primary key value x 1; for the data message 3, the data message 1 can be found according to the fact that the value of the field B in the data message 3 is the same as that of the field B in the data message 1, so that target data corresponding to the data message 3 is determined according to the primary key value x1 in the data message 1; for the data message 4, the target data corresponding to the data message 4 can be determined according to the primary key value x1 in the data message 1 by finding the data message 1 according to the fact that the value of the field C in the data message 4 is the same as that of the field C in the data message 3, and the value of the field B in the data message 3 is the same as that of the field B in the data message 1.
Data message 2 may be determined, and it may be determined that data message 2 corresponds to target data having a primary key value of 2 in the target data table based on the primary key value x2 it includes.
In addition, the data message 5 cannot currently determine which target data corresponds to the target data table, and needs to wait for the data message to be received again, for example, the data message 6 of the service line 2 (which includes the fields B and C with the values y2 and z2 when they are respectively) is received again, so that it can be determined that both the data message 5 and the data message 6 correspond to the target data with the primary key value of 2 in the target data table, and the specific process is the same as the principle of determining the target data corresponding to the data messages 1, 3 and 4, and will not be repeated.
Step 204, determining a consumption process corresponding to the target data, and sending the received data message to the consumption process corresponding to the target data, so that the consumption process writes the values of a plurality of fields included in the received data message into the target data table.
In the embodiment of the application, the method for determining the consumption process corresponding to the target data comprises the following steps: performing hash calculation on the value of a pre-designated data field included in target data, and determining a consumption process corresponding to a hash calculation result as a consumption process corresponding to the target data. For example, hash computation using field B will be specified in advance.
In the embodiment of the application, after the consumption process corresponding to the target data is determined, the received data message can be sent to the consumption process for processing.
For example, when the above data messages 1, 3, and 4 are all determined to correspond to the consumption process 1 after hash calculation using the value y1 of the field B, the three data messages are processed, namely: and writing the values of the fields included in the three data messages into a target data table, wherein the values of the fields A, B, C, D are sequentially x1, y1, z1 and q1 in target data with the primary key value of x1 in the target data table.
In addition, in the embodiment of the present application, in the case of high concurrency, after receiving the data message of each service line, the data message may be added to a message queue, and then another thread extracts the data message from the message queue one by one, so as to perform the operations in steps 203 and 204.
Therefore, in the embodiment of the present application, after receiving the data message from each service line, the received data message may be further added to a message queue. Before determining the target data in the target data table corresponding to the data message according to the values of the plurality of fields included in the received data message, the method further includes: reading the received data message from the message queue.
The following further exemplifies the embodiment of the present application, taking the bill message shipping message as an example:
suppose the following 4 waybill messages (including the waybill number and the waybill status) are received: ("1111", "1"), ("1111", "2"), ("2222", "1"), ("2222", "2"), and the following 2 shipping messages (including package number, site id, and manifest number) are received: ("1111-1", "A", "1111"), ("2222-1", "A", "2222").
By performing steps 203 and 204, two messages ("1111", "1"), ("1111", "2") and shipping messages ("1111-1", "a", "1111") of the manifest message are hashed into the same consumption process for single-threaded consumption, and the same manifest message ("2222", "1"), ("2222", "1") and shipping messages ("2222-1", "a", "2222") are hashed into the same consumption process for single-threaded consumption.
The two pieces of data finally and correctly written into the target data table are respectively (1111-1|A ","1111-1"," A ","1111","2 ") and (2222-1|A", "2222-1", "A", "2222", "2"), wherein "1111-1|A" and "2222-1|A" are the values of one main key field (package number |site id) formed by combination, and can be pre-designated as the main key field. In addition, other fields or combinations of fields that can uniquely identify one piece of data in the target data table may also be pre-designated as primary key fields.
It should be noted that: in the above-described process of writing the target data table, ("1111", "1") and ("2222", "1") are respectively covered by the later received ("1111", "2") and ("2222", "2").
In addition, the method shown in fig. 2 may be implemented in one docker, and the number of consumption processes in the docker is preferably the same as the cpu core number of the docker.
The data message processing method based on the multiple service lines in the embodiment of the application is described in detail, through the method, the received data messages of different service lines can be distributed to different consumption processes in a mode of combining and then hashing, and the consumption messages are processed through multi-process single-thread concurrency, so that the complexity of a multi-thread lock can be avoided, and the consumption performance can be effectively improved.
The embodiment of the application also provides a data message processing device based on a plurality of service lines, which is described in detail below with reference to fig. 3:
referring to fig. 3, fig. 3 is a schematic structural diagram of a data message processing apparatus according to an embodiment of the present application, which is applied to a data processing system, and includes a plurality of fields in a target data table in a data message of each service line; as shown in fig. 3, the apparatus includes:
a creation unit 301 for creating a plurality of consumption processes in advance;
a receiving unit 302 that receives data messages from each service line;
a determining unit 303, configured to determine target data in a target data table corresponding to the data message according to values of a plurality of fields included in the received data message;
and the hash unit 304 is configured to determine a consumption process corresponding to the target data, and send the received data message to the consumption process corresponding to the target data, so that the target consumption process writes the values of the multiple fields included in the received data message into the target data table.
In the apparatus shown in figure 3 of the drawings,
the receiving unit 302 is further configured to, after receiving the data message from each service line: adding the received data message into a message queue;
the determining unit 303 is further configured to, before determining the target data in the target data table corresponding to the data message according to the values of the plurality of fields included in the received data message: reading the received data message from the message queue.
In the apparatus shown in figure 3 of the drawings,
each piece of data in the target data table comprises a main key value which uniquely identifies the piece of data;
a pre-designated primary key field exists in a plurality of fields included in the data message of at least one service bar in the data messages of the service bars;
the data messages in different business lines realize direct association or indirect association through the same association field;
the determining unit 303 determines, according to the values of the plurality of fields included in the received data message, target data in a target data table corresponding to the data message, including:
if a pre-designated primary key field exists in a plurality of fields included in the data message, determining target data in a target data table corresponding to the received data message according to the value of the primary key field;
otherwise, the data message of any other business strip line with direct association or indirect association relation with the received data message is found out through the value of the association field in the received data message, and the target data in the target data table corresponding to the received data message is determined according to the value of the main key field of any other business strip line.
In the apparatus shown in figure 3 of the drawings,
the data messages in different service lines are indirectly associated through the same association field, including:
the data messages in the two business lines respectively have the same associated fields with the data messages in the other business line, so that the first indirect association of the data messages in the two business lines is realized;
or alternatively, the process may be performed,
the data messages in the two business lines are respectively and indirectly related to the data messages in the other business line;
or alternatively, the process may be performed,
the data messages in two traffic lanes are directly associated and first indirectly associated, respectively, with the data messages in the other traffic lane.
In the apparatus shown in figure 3 of the drawings,
the hash unit 304 is configured to, when determining a consumption process corresponding to the target data:
performing hash calculation on the value of a pre-designated data field included in target data, and determining a consumption process corresponding to a hash calculation result as a consumption process corresponding to the target data.
The embodiment of the application also provides an electronic device, as shown in fig. 4, an electronic device 400 includes: at least one processor 401, and a memory 402 connected to the at least one processor 401 through a bus; the memory 402 stores one or more computer programs executable by the at least one processor 401; the at least one processor 401, when executing the one or more computer programs, implements the steps of the method as shown in fig. 2.
Embodiments of the present application also provide a computer readable storage medium storing one or more computer programs that when executed by a processor implement the method of fig. 2.
The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the application.
Claims (8)
1. A data message processing method based on a plurality of business strip lines is applied to a data processing system, and is characterized in that the data message of each business strip line comprises a plurality of fields in a target data table; the method comprises the following steps:
pre-creating a plurality of consumption processes;
receiving data messages from each service stripe;
determining target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message;
determining a consumption process corresponding to the target data, and sending the received data message to the consumption process corresponding to the target data, so that the consumption process writes the values of a plurality of fields included in the received data message into the target data table;
wherein each piece of data in the target data table comprises a primary key value uniquely identifying the piece of data;
a pre-designated primary key field exists in a plurality of fields included in the data message of at least one service bar in the data messages of the service bars;
the data messages in different business lines realize direct association or indirect association through the same association field;
determining target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message, wherein the method comprises the following steps:
if a main key field exists in a plurality of fields included in the data message, determining target data in a target data table corresponding to the received data message according to the value of the main key field;
otherwise, the data message of any other business strip line with direct association or indirect association relation with the received data message is found out through the value of the association field in the received data message, and the target data in the target data table corresponding to the received data message is determined according to the value of the main key field of any other business strip line.
2. The method according to claim 1, wherein;
after receiving the data message from each traffic stripe, further comprising: adding the received data message into a message queue;
before determining the target data in the target data table corresponding to the data message according to the values of the fields included in the received data message, the method further includes: reading the received data message from the message queue.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the indirect association of the data messages in the different business lines through the same association field comprises the following steps:
the data messages in the two business lines respectively have the same associated fields with the data messages in the other business line, so that the first indirect association of the data messages in the two business lines is realized;
or alternatively, the process may be performed,
the data messages in the two business lines are respectively and indirectly related to the data messages in the other business line;
or alternatively, the process may be performed,
the data messages in two traffic lanes are directly associated and first indirectly associated, respectively, with the data messages in the other traffic lane.
4. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the method for determining the consumption process corresponding to the target data comprises the following steps:
performing hash calculation on the value of a pre-designated data field included in target data, and determining a consumption process corresponding to a hash calculation result as a consumption process corresponding to the target data.
5. A data message processing device based on a plurality of business strip lines, which is applied to a data processing system, and is characterized in that the data message of each business strip line comprises a plurality of fields in a target data table; the device comprises:
a creation unit for creating a plurality of consumption processes in advance;
a receiving unit for receiving data messages from each service line;
a determining unit, configured to determine target data in a target data table corresponding to the data message according to values of a plurality of fields included in the received data message;
the hash unit is used for determining a consumption process corresponding to the target data, and sending the received data message to the consumption process corresponding to the target data so that the target consumption process writes the values of a plurality of fields included in the received data message into the target data table;
wherein, the liquid crystal display device comprises a liquid crystal display device,
each piece of data in the target data table comprises a main key value which uniquely identifies the piece of data;
a pre-designated primary key field exists in a plurality of fields included in the data message of at least one service bar in the data messages of the service bars;
the data messages in different business lines realize direct association or indirect association through the same association field;
the determining unit determines target data in a target data table corresponding to the data message according to the values of a plurality of fields included in the received data message, and includes:
if a main key field exists in a plurality of fields included in the data message, determining target data in a target data table corresponding to the received data message according to the value of the main key field;
otherwise, the data message of any other business strip line with direct association or indirect association relation with the received data message is found out through the value of the association field in the received data message, and the target data in the target data table corresponding to the received data message is determined according to the value of the main key field of any other business strip line.
6. The apparatus of claim 5, wherein the apparatus comprises;
the receiving unit, after receiving the data message from each service line, is further configured to: adding the received data message into a message queue;
the determining unit is further configured to, before determining the target data in the target data table corresponding to the data message according to the values of the plurality of fields included in the received data message: reading the received data message from the message queue.
7. An electronic device, comprising: at least one processor, and a memory connected to the at least one processor via a bus; the memory stores one or more computer programs executable by the at least one processor; the method steps of any of claims 1-4 are implemented when said at least one processor executes said one or more computer programs.
8. A computer readable storage medium storing one or more computer programs which when executed by a processor implement the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910992376.8A CN112685191B (en) | 2019-10-18 | 2019-10-18 | Message processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910992376.8A CN112685191B (en) | 2019-10-18 | 2019-10-18 | Message processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112685191A CN112685191A (en) | 2021-04-20 |
| CN112685191B true CN112685191B (en) | 2023-09-22 |
Family
ID=75444887
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910992376.8A Active CN112685191B (en) | 2019-10-18 | 2019-10-18 | Message processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685191B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9130860B1 (en) * | 2014-10-09 | 2015-09-08 | Splunk, Inc. | Monitoring service-level performance using key performance indicators derived from machine data |
| CN108536544A (en) * | 2018-03-21 | 2018-09-14 | 微梦创科网络科技（中国）有限公司 | Consuming method, device, server based on database message queue and medium |
| CN109144683A (en) * | 2017-06-28 | 2019-01-04 | 北京京东尚科信息技术有限公司 | Task processing method, device, system and electronic equipment |
| CN109325055A (en) * | 2018-08-02 | 2019-02-12 | 阿里巴巴集团控股有限公司 | The screening of business association tables of data and checking method, device, electronic equipment |
| CN109726237A (en) * | 2018-12-13 | 2019-05-07 | 浙江邦盛科技有限公司 | A kind of association complementing method for multichannel real-time streaming data |
| CN110209677A (en) * | 2018-02-06 | 2019-09-06 | 北京京东尚科信息技术有限公司 | The method and apparatus of more new data |
-
2019
- 2019-10-18 CN CN201910992376.8A patent/CN112685191B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9130860B1 (en) * | 2014-10-09 | 2015-09-08 | Splunk, Inc. | Monitoring service-level performance using key performance indicators derived from machine data |
| CN109144683A (en) * | 2017-06-28 | 2019-01-04 | 北京京东尚科信息技术有限公司 | Task processing method, device, system and electronic equipment |
| CN110209677A (en) * | 2018-02-06 | 2019-09-06 | 北京京东尚科信息技术有限公司 | The method and apparatus of more new data |
| CN108536544A (en) * | 2018-03-21 | 2018-09-14 | 微梦创科网络科技（中国）有限公司 | Consuming method, device, server based on database message queue and medium |
| CN109325055A (en) * | 2018-08-02 | 2019-02-12 | 阿里巴巴集团控股有限公司 | The screening of business association tables of data and checking method, device, electronic equipment |
| CN109726237A (en) * | 2018-12-13 | 2019-05-07 | 浙江邦盛科技有限公司 | A kind of association complementing method for multichannel real-time streaming data |
Non-Patent Citations (2)
| Title |
|---|
| 李艳春 ; 李新 ; 焦文彬 ; .分布式信息系统中数据交换平台设计与实现.计算机工程与设计.2012,(07),全文. * |
| 王俊 ; 胡牧 ; 孟庆强 ; 顾学海 ; .移动数据同步冲突预处理与消解策略研究.信息技术.2017,(第12期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112685191A (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410045B (en) | Parallel chain consensus method, equipment and storage medium | |
| CN104881494B (en) | The methods, devices and systems synchronous with Redis server progress data | |
| US8468313B2 (en) | Asynchronous replication with write concurrency grouping | |
| CN110673932B (en) | Multi-database log with multi-project transaction support | |
| CN108830777B (en) | Techniques for fully synchronizing execution threads | |
| US20130041869A1 (en) | Real Time Data Replication | |
| CN103514053A (en) | Shared-memory-based method for conducting communication among multiple processes | |
| CN105630731A (en) | Network card data processing method and device in multi-CPU (Central Processing Unit) environment | |
| WO2017041638A1 (en) | Log data processing method and device | |
| US20210382721A1 (en) | Central processor-coprocessor synchronization | |
| RU2654144C1 (en) | Redo logging for partitioned data set in memory | |
| US10732841B2 (en) | Tracking ownership of memory in a data processing system through use of a memory monitor | |
| CN108765142B (en) | Electronic device, multi-blockchain integration control method, and computer-readable storage medium | |
| CN105607968B (en) | A kind of incremental backup method and equipment | |
| JP2004252986A (en) | System and method of distributing replication command | |
| US20160124855A1 (en) | Processing an input/ output operation request | |
| US8046780B1 (en) | Efficient processing of assets with multiple data feeds | |
| US8850074B2 (en) | Data synchronization method | |
| CN107277022B (en) | Process marking method and device | |
| CN112685191B (en) | Message processing method and device | |
| US20110320604A1 (en) | Allocating unique identifiers | |
| CN110737504A (en) | deep learning model training fault-tolerant method, system, terminal and storage medium | |
| TWI385529B (en) | Bus system and operation method thereof | |
| CN105391755A (en) | Method and device for processing data in distributed system, and system | |
| CN107832121B (en) | Concurrency control method applied to distributed serial long transactions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |