CN110175084B

CN110175084B - Data change monitoring method and device

Info

Publication number: CN110175084B
Application number: CN201910271980.1A
Authority: CN
Inventors: 刘晖
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-04-04
Filing date: 2019-04-04
Publication date: 2023-04-25
Anticipated expiration: 2039-04-04
Also published as: CN110175084A

Abstract

The embodiment of the specification provides a method and a device for monitoring data change, wherein the method comprises the following steps: acquiring service data generated by a first service node when processing a service; detecting whether the service data accords with a predetermined historical data characteristic detection rule; wherein the historical data feature detection rule is determined based on historical service data generated by the first service node; if the detection result indicates that the service data does not accord with the historical data characteristic detection rule, sending prompt information to a second service node; the prompt information is used for indicating the change of the data characteristics of the service data generated by the first service node; the second service node is a downstream node of the first service node.

Description

Data change monitoring method and device

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a method and an apparatus for monitoring data changes.

Background

With the rapid development of the internet and information technology, more and more businesses need to be transacted by means of business systems, such as online transaction business, transfer business, payment business, etc. In general, when a service system performs service processing, a plurality of service nodes in the service system are required to cooperatively perform the processing of the service.

With the continuous expansion of the service scale, a certain service node may adjust the data content such as the service mode, for example, a new service mode is added, or certain parameters of a certain service are adjusted, but the downstream service node of the service node cannot learn the change condition of the upstream node, and when the service is processed, the service is still processed according to the original service mode, thereby causing the service processing to fail.

Therefore, how to monitor the service modes of each service node and reduce the occurrence of service failure caused by the monitoring, is a technical problem that needs to be solved currently.

Disclosure of Invention

An object of the embodiments of the present disclosure is to provide a method and an apparatus for monitoring data change, by detecting whether service data generated by a first service node when processing a service accords with a predetermined historical data feature detection rule, to monitor whether a service data feature of the first service node changes, so as to determine whether a service mode of the first service node is adjusted, and when it is detected that the service data feature of the first service node changes, to timely notify a second service node, where the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

In order to solve the above technical problems, the embodiments of the present specification are implemented as follows:

the embodiment of the specification provides a method for monitoring data change, which comprises the following steps:

acquiring service data generated by a first service node when processing a service;

detecting whether the service data accords with a predetermined historical data characteristic detection rule; wherein the historical data feature detection rule is determined based on historical service data generated by the first service node;

if the detection result indicates that the service data does not accord with the historical data characteristic detection rule, a prompt message is sent to a second service node; the prompt information is used for indicating that the data characteristics of the service data generated by the first service node are changed; the second service node is a downstream node of the first service node.

The embodiment of the specification also provides a device for monitoring data change, which comprises:

the first acquisition module is used for acquiring service data generated by the first service node when the service is processed;

the detection module is used for detecting whether the service data accords with a predetermined historical data characteristic detection rule; wherein the historical data feature detection rule is determined based on historical service data generated by the first service node;

The sending module is used for sending prompt information to a second service node if the detection result indicates that the service data does not accord with the historical data characteristic detection rule; the prompt information is used for indicating that the data characteristics of the service data generated by the first service node are changed; the second service node is a downstream node of the first service node.

The embodiment of the specification also provides a monitoring device for data change, which comprises:

a processor; and

a memory arranged to store computer executable instructions that, when executed, cause the processor to:

The present description also provides a storage medium for storing computer-executable instructions that, when executed, implement the following:

According to the technical scheme, whether the service data characteristics of the first service node are changed or not is monitored by detecting whether the service data generated by the first service node when the service is processed accords with a predetermined historical data characteristic detection rule, whether the service mode of the first service node is adjusted or not can be judged according to the detection rule, and a second service node is timely notified when the change of the service data characteristics of the first service node is detected, wherein the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

Drawings

In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings without inventive effort for a person of ordinary skill in the art.

FIG. 1 is a flowchart of a method for monitoring data changes according to an embodiment of the present disclosure;

FIG. 2 is a second flowchart of a method for monitoring data changes according to an embodiment of the present disclosure;

FIG. 3 is a flowchart of a method for determining a historical data feature detection rule in the method for monitoring data change according to the embodiment of the present disclosure;

FIG. 4 is a schematic flow chart of determining historical data characteristics in the method for monitoring data change according to the embodiment of the present disclosure;

FIG. 5 is a third flowchart of a method for monitoring data changes according to the embodiment of the present disclosure;

FIG. 6 is a fourth flowchart of a method for monitoring data changes according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a module composition of a monitoring device for data change according to an embodiment of the present disclosure;

Fig. 8 is a schematic structural diagram of a monitoring device for data change according to an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions in the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

According to the monitoring method for data change provided by the embodiment of the specification, whether the service mode of the upstream service node is adjusted is achieved by detecting whether the data characteristics of the service data generated by the upstream service node when the service is processed are changed, so that the downstream service node can acquire the adjustment condition of the service mode of the upstream service node in time, and then corresponding adjustment is made, and the occurrence of service processing faults is reduced.

The method for monitoring data change provided in the embodiment of the present disclosure is applied to a device for monitoring data change, where the device for monitoring data change is independent of devices other than each service node in a service system, and may be one device node in the service system or may be other than a device, and the embodiment of the present disclosure is not limited to this.

The main execution body of the method for monitoring data change provided in the embodiments of the present disclosure is the device for monitoring data change, and specifically, is a device for monitoring data change installed on the device.

Fig. 1 is one of the method flowcharts of the method for monitoring data changes according to the embodiment of the present disclosure, and the method shown in fig. 1 at least includes the following steps:

step 102, obtaining service data generated by the first service node when processing the service.

The service may be any type of service handled by a service system, for example, a transfer service, a recharging service, a payment service, etc. The service data may include all data related to the service, such as a service type of the service, a service status of the service, a service party, a service order number, and the like.

For example, if the service is a transfer service, the service data may include: service type, identification information of users of both sides of the transfer, transfer amount, order number, transfer time, transfer mode and the like.

Specifically, in the step 102, service data generated by the first service node when processing the service is obtained, which at least includes the following two cases:

First, the generated service data is acquired from the first service node when the first service node processes the service.

In this case, it may be that the monitoring means for data change acquires a piece of service data from the first service node every time the first service node generates the piece of service data.

Secondly, acquiring service data from a database; wherein, the database stores the business data generated by the first business node when processing business.

In this case, when the first service data processes the service, the generated service data is stored in the database, and the monitoring device for data change only needs to acquire the service data corresponding to the first service node from the database. Specifically, the device for changing data can acquire service data from the database according to a set period.

The database may be mysql database, hbase database, oceanBase database, etc.

104, detecting whether the service data accords with a predetermined historical data characteristic detection rule; wherein the historical data feature detection rule is determined based on historical service data generated by the first service node.

Before executing the method provided in the embodiments of the present disclosure, the historical data feature detection rule corresponding to the first service node may be determined according to the historical service data generated by the first service node stored in the database. Typically for a piece of traffic data, it includes fields of an enumerable type and fields of a non-enumerable type. The field of the enumerated type refers to that the data value corresponding to the field is a finite sequence, that is, the data value can be enumerated completely. For example, the data values corresponding to Monday, tuesday, friday, saturday, and Sunday are finite series of data, and thus, the Tuesday data may be enumerated in type; also for example, if the above-mentioned field is a service state, the general service state includes two kinds of success and failure, and the data is in a finite sequence, so the service state belongs to the field of the enumerated type.

Accordingly, a field of a non-enumeratable type refers to a data value corresponding to the field being an infinite sequence, i.e. the data values cannot be all enumerated. For example, for a field of order numbers, each service corresponds to an order number, and as long as the service system is continuously processing the service, all order numbers cannot be listed, so the order numbers belong to a field of non-enumeration type.

In this embodiment of the present disclosure, the above-mentioned historical data feature detection rule is generally determined based on a combination of data values corresponding to the fields of the enumerable type and feature description information of data values corresponding to the fields of the non-enumerable type in the historical service data. I.e. if the history data includes an enumerable type field and a non-enumerable type field; each historical data characteristic detection rule is formed by combining a historical data value corresponding to each field of the enumerated type and historical data characteristic description information of data corresponding to each field of the non-enumerated type.

For ease of understanding, the following examples are presented.

For example, in one embodiment, the above-mentioned historical data feature detection rule may be expressed as follows:

Service type: transferring accounts; service status: success; order information: order length 12 digits. Of course, a plurality of history data feature detection rules may be predetermined according to the history service data, and only one of the history data feature detection rules is described as an example, and the embodiment of the present disclosure is not limited thereto.

Specifically, after the service data generated by the first service node is obtained, it is required to detect whether the feature information corresponding to each field in the service data meets the above historical data feature detection rule.

Step 106, if the detection result indicates that the service data does not accord with the history data feature detection rule, sending prompt information to a second service node; the prompt information is used for indicating that the data characteristics of the service data generated by the first service node are changed; the second service node is a downstream node of the first service node.

In the embodiment of the present disclosure, if the detection result indicates that the service data accords with the historical data feature detection rule, it is considered that, compared with the historical service data, the data feature of the service data generated by the first service node is unchanged, that is, a developer of the first service node does not adjust the service mode of the first service node (such as adding a new service mode, etc.), and then the flow is ended; if the detection result indicates that the service data does not accord with the historical data feature detection rule, the data feature of the service data generated by the first service node is considered to be changed, namely the service mode of the first service node is adjusted, and in this case, in order to facilitate the downstream node of the first service node to correspondingly adjust the service mode in time, thereby ensuring the normal processing of the service, prompt information needs to be sent to the second service node (downstream node of the first service node).

In the implementation, the prompt information can be sent to the developer of the second service node in a mail mode, a short message mode, a third party instant messaging message mode and the like. Specifically, the prompt information needs to carry data content with characteristic change of the first service node, so that a worker of the second service node can make corresponding adjustment on the second service node according to the current service mode of the first service node in time, and normal processing of service of the service system is guaranteed.

According to the monitoring method for data change provided by the embodiment of the specification, whether the data characteristics of the service data generated by the first service node change or not can be determined by detecting whether the service data generated by the first service processing node accords with the predetermined historical data characteristic detection rule, and further whether the service mode of the first service node is adjusted or not can be judged through the change of the data characteristics of the service data of the first service node, so that when the service mode of the first service node is determined to be adjusted, a downstream node (a second service node) of the first service node can be timely notified, and a developer of the second service node can timely and correspondingly adjust the second service node, so that normal processing of service system service is ensured.

In the implementation, in the step 104, the detecting whether the service data meets the predetermined historical data feature detection rule specifically includes:

determining the data characteristics of the data values corresponding to the fields in the service data; and respectively matching the data characteristics corresponding to the service data with each historical data characteristic detection rule to determine whether the service data accords with the historical data characteristic detection rules.

In specific implementation, when determining the data characteristics corresponding to the service data, an enumerable type field and a non-enumerable type field in the service data may be determined first. For the field of the enumeration type, the data value corresponding to the field is directly used as the data feature corresponding to the field, for example, for the field of the "service state", which belongs to the field of the enumeration type, if the data value corresponding to the "service state" in the service data is "success", the data feature corresponding to the service state is determined as "success". And aiming at the field of the non-enumeratable type in the service data, determining the data characteristic description information of the data value corresponding to the field as the corresponding data characteristic. For example, if the "order number" in the service data is "12787565" for a field of "order number", the "length of 8 digits" may be used as the data feature corresponding to the "order number" field. Of course, the data characteristic description information determination rule corresponding to each field may be preset, for example, the data characteristic description information determination rule may indicate that the length information of the data value corresponding to the field is used as the corresponding data characteristic description information thereof, or the like.

After determining the data characteristics corresponding to each field in the service data, respectively matching the data characteristics with each predetermined historical data characteristic detection rule, and if a certain historical data characteristic detection rule is matched with the data characteristics corresponding to the service data, considering that the service data accords with the historical data characteristic detection rule, namely, compared with the historical service data, the data characteristics of the service data are unchanged.

For ease of understanding, the following examples are presented.

For example, in one embodiment, the data characteristics corresponding to the service data are:

service type: transferring accounts; service status: success; order number: ten digits;

the predetermined historical data feature detection rules comprise the following three rules:

rule 1, service type: transferring accounts; service status: failure; order number: consists of ten digits;

rule 2, service type: recharging; service status: success; order number: consists of 12 digits;

rule 3, service type: transferring accounts; service status: success; order number: consists of 12 digits;

the data characteristics corresponding to the service data are respectively matched with the rule 1, the rule 2 and the rule 3, and the data characteristics are found to be not matched with the rule 1, the rule 2 and the rule 3 through matching, so that the data characteristics of the service data are considered to be changed compared with the historical service data, namely, a developer of the first service node adjusts the service mode, and in order to facilitate the developer of the second service node to make corresponding adjustment on the service in time, prompt information needs to be sent to the second service node.

Specifically, in the embodiment of the present disclosure, if it is detected that the data characteristics of the service data generated by the first service node change, in addition to sending prompt information to the second service node, a data characteristic detection rule for the service data needs to be determined and used as a historical data characteristic detection rule, so when the next time the first service node generates the service data with the same data characteristic again, since the prompt information has been sent to the second service node for the data with the same data characteristic, the second service node will not send a prompt again, so as to avoid repeated prompt, and increase the workload of the developer of the second service node. Therefore, if the detection result indicates that the service data does not meet the historical data feature detection rule, the method provided in the embodiment of the present disclosure further includes the following steps:

determining a data characteristic detection rule corresponding to the service data; adding the data characteristic detection rule into a historical data characteristic detection rule set; wherein the set of historical data feature detection rules includes each historical data feature detection rule.

In the embodiment of the present disclosure, the data feature detection rule corresponding to the service data may be determined according to the feature value of the data corresponding to each field in the service data. In a specific implementation manner, if the service data includes an enumerated type field and a non-enumerated type field;

Correspondingly, the data feature detection rule corresponding to the determined service data specifically includes:

determining a data value corresponding to an enumeration type field in the service data, and determining data characteristic description information corresponding to a non-enumeration type field in the service data; and combining the data values and the data characteristic description information to obtain a data characteristic detection rule corresponding to the service data.

In this embodiment of the present disclosure, when determining a data feature detection rule corresponding to service data, an enumeration type field and a non-enumeration type field in the service data are first determined, and then, for the non-enumeration type field, the data feature corresponding to the field needs to be determined according to a preset data feature description information determination rule. For example, the data feature description information determining rule indicates that the data feature of the field needs to be described from the constituent length of the field, and then the length information corresponding to the field is determined as the data feature description information corresponding to the field; and finally, combining the data value of the field of the enumeration type and the data characteristic description information of the field of the non-enumeration type together to serve as a data characteristic detection rule corresponding to the service data.

In addition, it should be noted that, in the embodiment of the present disclosure, the service data generated by the acquired first service node may be directly service data in a key-value form, where the key represents each field in the service data, and the value represents a data value corresponding to each field, where in this case, the data value corresponding to each field may be directly acquired from the service data. In some cases, the acquired service data may not be in the form of key-value, and at this time, format conversion is first required to be performed on the service data to obtain the service data in the form of key-value, and then the data value corresponding to each field is determined.

Of course, in the implementation, if the acquired service data is not the key-value form service data, each field and the value corresponding to each field may be directly extracted from the data. In the embodiment of the present specification, as long as the fields and the corresponding data values thereof can be obtained finally, the embodiment of the present specification does not limit the specific implementation procedure.

For ease of understanding, the following will exemplify the specific procedure for determining the data feature detection rule described above.

For example, one piece of acquired service data is:

service type	Service status	Transfer amount of money	Order number
				Transfer of money	Successful	100.00 yuan	1267996413

The fields corresponding to the service data include: "service type", "service status", "transfer amount" and "order number". Of these fields, "service type" and "service status" belong to an enumerable type field, and "transfer amount" and "order number" belong to a non-enumerable type field. Therefore, when determining the data feature detection rule, the data values of the "service type" and the "service state" may be directly used as the data features, and for the "transfer amount" and the "order number", the data feature description information corresponding to the "transfer amount" needs to be determined, for the "transfer amount", the decimal point may be followed by two bits as the data feature data description information corresponding to the decimal point, for the "order number", the data feature description information corresponding to the "order number" may be formed by a number with a length of 10 bits, and then, the data features corresponding to the fields are combined together, thereby obtaining the data feature detection rule corresponding to the service data.

Thus, for this traffic data, its corresponding data feature detection rule is:

Service type: transferring accounts; service status: success; amount of money: two decimal places after the decimal point; order number: ten digits.

Fig. 2 is a second flowchart of a method for monitoring data changes according to an embodiment of the present disclosure, where the method shown in fig. 2 at least includes the following steps:

step 202, obtaining service data generated by a first service node when processing a service.

Step 204, detecting that the service data is in accordance with a history data feature detection rule in a predetermined history data feature detection rule set; if yes, ending; otherwise, step 206 is performed.

And step 206, sending prompt information to the second service node.

Step 208, determining a data value corresponding to the field of the enumeration type in the service data, and determining data feature description information corresponding to the field of the non-enumeration type in the service data.

And 210, combining the data values and the data characteristic description information to obtain a data characteristic detection rule corresponding to the service data.

Step 212, adding the data feature detection rule as a historical data feature detection rule to the set of historical data feature detection rules.

In addition, in the embodiment of the present disclosure, in order to facilitate the detection of whether the service data generated by the first service node meets the predetermined historical data feature detection rule, the historical data feature detection rule needs to be determined according to the service data generated by the first service node, so before the service data generated by the first service node during the service processing is acquired, the method provided in the embodiment of the present disclosure further includes the following steps one and two;

step one, acquiring historical service data corresponding to a first service node;

and step two, determining a historical data characteristic detection rule based on the historical service data.

In practice, all the historical service data generated by the first service node may be obtained from the database. In particular, the obtained historical traffic data may be all the historical traffic data accumulated so far from the first traffic node to start processing traffic.

In the embodiment of the present disclosure, in the second step, a history data feature detection rule is determined based on the history service data, and specifically includes the following steps (1), step (2), and step (3);

step (1), performing format conversion on the historical service data to obtain the historical service data in a key-value form; wherein, the key characterizes the field, the value characterizes the data value corresponding to the above-mentioned field;

Step (2), determining the data characteristics corresponding to the fields according to the data values corresponding to the fields;

and (3) determining a historical data characteristic detection rule based on the data characteristics corresponding to the fields.

In one embodiment, all of the historical service data generated by the first service node may be obtained from the message bus by a feature factory in the monitoring device for data changes. In order to determine the historical data feature detection rule, in this embodiment of the present disclosure, the acquired DTO needs to be converted into service data in a key-value form, where the key represents each field in the service data, such as a field of "service type", "service state", "service amount", "service party", and the like, and the value represents a data value corresponding to each field.

In general, the acquired service data includes a plurality of fields, and when the acquired service data is converted into data in a key-value format, the fields are converted into key values and value values. However, in practical applications, it often occurs that a plurality of extension fields are nested in some fields, and for this case, each field in the extension fields needs to be converted into a corresponding key-value form, which will be illustrated for the sake of understanding.

For example, in one embodiment, the acquired historical business data is as follows:

in the above service data, the column of "extension information" belongs to the case of nesting fields, and when converting the column of data into a key-value form, each field to be nested needs to be converted into a corresponding key-value form.

And converting the service data into a key-value format, wherein the converted historical service data is shown in the following table:

key	value
		order number	1231231
Total amount of sum	123
		Status of	Successful
Type(s)	Recharging method
		Expansion information: payer identification	12312312
Expansion information: operating system	Android
		Expansion information: expansion information: aa	xxx
Expansion information: expansion information: bb [0 ]]	111
		Expansion information: expansion information: bb [1 ]]	222
Expansion information: expansion information: bb [2 ]]	333

It should be noted that, in the embodiment of the present specification, the above nested fields may include two types of data, one type is a single-column data set (List), and one type is a double-column data set (Map), such as "payer identifier": "12312312" belongs to List, "bb" is [ "111", "222", "333" ] belongs to Map, and the key-value form converted from List class and Map class nested fields is shown in the above table.

Of course, the description is intended to be illustrative only and is not to be construed as limiting the embodiments herein.

Specifically, in the step (2), the determining the data feature corresponding to each field according to the data value corresponding to each field specifically includes:

if the field is an enumerated field, determining a data value corresponding to the field as a data characteristic corresponding to the field; if the field is a field of a non-enumeration type, determining data characteristic description information corresponding to the field according to the data value corresponding to each field, and determining the data characteristic description information corresponding to the field as the data characteristic corresponding to the field.

In a specific implementation, an enumeration generator in the data change monitoring device may be used to determine an enumeration field and a non-enumeration field in the historical service data, and distinguish the enumeration field and the corresponding data value in the service data, where the data value corresponds to the non-enumeration field.

When it should be noted that, for the fields of the enumerable type and the fields of the non-enumerable type, how to determine the corresponding feature information is described in the foregoing, reference may be made to the foregoing description specifically, and details are not repeated here.

In the step (3), the determining a historical data feature detection rule based on the data features corresponding to the fields specifically includes:

Combining the data characteristics corresponding to each field in each piece of historical service data to obtain a data characteristic combination result; and respectively determining each data characteristic combination result as a historical data characteristic detection rule.

In order to facilitate understanding of the technical solutions herein, the following examples will be given.

For example, the three pieces of obtained historical service data are respectively recorded as historical service data 1, historical service data 2 and historical service data 3, the data features corresponding to the fields in the historical service data 1 are combined to obtain a data feature combination result 1, the data features corresponding to the fields in the historical service data 2 are combined to obtain a data feature combination result 2, the data features corresponding to the fields in the historical service data 3 are combined to obtain a data feature combination 3, when the historical data feature detection rule is determined according to the data feature combination result, the data feature combination result 1 is determined to be a historical data feature detection rule, the data feature combination result 2 is determined to be a historical data feature detection rule, and the data feature combination result 3 is determined to be a historical data feature detection rule.

Wherein, a piece of historical service data can be understood as service data generated by the first service node processing a service.

Of course, in the implementation, there may be a case where the history data feature detection rules determined by the plurality of pieces of service data are the same, and in this case, it is necessary to reject the repeated history data feature detection rules.

Fig. 3 is a flowchart of a method for determining a historical data feature detection rule in the method for monitoring data change provided in the embodiment of the present disclosure, where the method shown in fig. 3 at least includes the following steps:

step 302, obtaining all historical service data corresponding to the first service processing node.

And step 304, performing format conversion on the historical service data to obtain the historical service data in a key-value form.

In this embodiment of the present disclosure, the key represents each field in the service data, and the value represents a data value corresponding to each field.

In step 306, an enumerable type field and a non-enumerable type field in the historical business data are determined.

Step 308, for the field of the enumerated type, determining the data value corresponding to the field as the data feature corresponding to the field, for the field of the non-enumerated type, determining the data feature description information corresponding to the field according to the data value corresponding to the field, and determining the data feature description corresponding to the field as the data feature corresponding to the field.

Step 310, the data features corresponding to the fields in each piece of historical service data are combined to obtain a data feature combination result.

At step 312, each data feature combination is determined as a historical data feature detection rule.

Fig. 4 is a flowchart corresponding to the flowchart of the method shown in fig. 3, where fig. 4 is a flowchart showing a method for determining a historical data feature detection rule in the method for monitoring data change according to the embodiment of the present disclosure.

For the flow chart shown in fig. 4, the feature factory acquires the historical service data generated by the first service node from the message bus, converts the historical service data into service data in a key-value format, then determines the field of an enumeration type and the field of a non-enumeration type in the service data through an enumeration generator, determines feature values of the field of the enumeration type and feature description information of the feature values of the field of the non-enumeration type, and combines the feature values of the field of the enumeration type and the feature description information of the field of the non-enumeration type together to obtain a historical data feature detection rule.

In the method provided in the embodiment of the present disclosure, when service data generated by the first service node during service processing is obtained, corresponding service data may be directly obtained from the first service node, or corresponding service data may be obtained from a database, which will be described in detail below.

Fig. 5 is a third flowchart of a method for monitoring data changes according to an embodiment of the present disclosure, where the method shown in fig. 5 at least includes the following steps:

step 502, service data generated by the first service node when processing the service is obtained from the first service node.

Step 504, determining the data characteristics of the data values corresponding to the fields in the service node.

Step 506, detecting whether a history data feature detection rule matched with the data feature of the service data exists in each predetermined history data feature detection rule set, and ending if the history data feature detection rule exists; otherwise, step 508 is performed.

And step 508, sending prompt information to the second node, wherein the prompt information is used for indicating that the data characteristics of the service data generated by the first service node are changed.

Step 510, determining a data feature detection rule corresponding to the service data.

Step 512, adding the data feature detection rule as a historical data feature detection rule to the historical data feature rule detection set.

Fig. 6 is a fourth flowchart of a method for monitoring data changes according to an embodiment of the present disclosure, where the method shown in fig. 6 at least includes the following steps:

Step 602, obtaining service data of a first service node from a database; wherein, the database stores all the business data generated by the first business node when processing business.

In the implementation, the service data generated by the first service node in the period can be obtained from the database according to the set period.

Step 604, determining the data characteristics of the data values corresponding to the fields in the service node.

Step 606, detecting whether a history data feature detection rule matched with the data feature of the service data exists in each predetermined history data feature detection rule set, and ending if the history data feature detection rule exists; otherwise, step 608 is performed.

Step 608, sending a prompt message to the second node, where the prompt message is used to indicate that the data feature of the service data generated by the first service node changes.

Step 610, determining a data feature detection rule corresponding to the service data.

Step 612, adding the data feature detection rule as a historical data feature detection rule to the historical data feature rule detection set.

According to the monitoring method for data change provided by the embodiment of the specification, whether the service data characteristics of the first service node are changed is monitored by detecting whether the service data generated by the first service node when the service is processed accords with the predetermined historical data characteristic detection rule, whether the service mode of the first service node is adjusted can be judged according to the detection rule, and when the change of the service data characteristics of the first service node is detected, a second service node is timely notified, wherein the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

Corresponding to the method for monitoring data change provided in the embodiment of the present disclosure, based on the same concept, the embodiment of the present disclosure further provides a device for monitoring data change, configured to execute the method provided in the embodiment of the present disclosure, fig. 7 is a schematic block diagram of the device for monitoring data change provided in the embodiment of the present disclosure, where the device shown in fig. 7 includes:

a first obtaining module 702, configured to obtain service data generated by a first service node when processing a service;

a detection module 704, configured to detect whether the service data accords with a predetermined historical data feature detection rule; wherein the historical data feature detection rule is determined based on historical service data generated by the first service node;

a sending module 706, configured to send a prompt message to the second service node if the detection result indicates that the service data does not conform to the historical data feature detection rule; the prompt information is used for indicating the change of the data characteristics of the service data generated by the first service node; the second service node is a downstream node of the first service node.

Optionally, if the detection result indicates that the service data does not meet the historical data feature detection rule, the apparatus provided in the embodiment of the present disclosure further includes:

The first determining module is used for determining a data characteristic detection rule corresponding to the service data;

the adding module is used for adding the data characteristic detection rule into the historical data characteristic detection rule set; wherein each historical data characteristic detection rule is included in the historical data characteristic detection rule set.

Optionally, the detection module includes:

the first determining unit is used for determining the data characteristics of the data values corresponding to the fields in the service data;

and the matching unit is used for respectively matching the data characteristics corresponding to the service data with each historical data characteristic detection rule so as to determine whether the service data accords with the historical data characteristic detection rules.

Optionally, if the service data includes an enumerated field and a non-enumerated field;

the determining module includes:

the second determining unit is used for determining a data value corresponding to an enumeration type field in the service data and determining data characteristic description information corresponding to a non-enumeration type field in the service data;

and the combination unit is used for combining the data value and the data characteristic description information to obtain a data characteristic detection rule corresponding to the service data.

Optionally, if the history service data includes an enumerated field and a non-enumerated field;

each of the above-mentioned history data feature detection rules is formed by combining the history data value corresponding to each field of the enumerated type and the history data feature description information corresponding to each field of the non-enumerated type.

Optionally, the first obtaining module includes:

a first obtaining unit, configured to obtain, when the first service node processes a service, generated service data from the first service node;

or, the second acquisition unit is used for acquiring the service data from the database; wherein, the database stores the business data generated by the first business node when processing business.

Optionally, an apparatus provided in an embodiment of the present disclosure further includes:

the second acquisition module is used for acquiring historical service data corresponding to the first service node;

and the second determining module is used for determining a historical data characteristic detection rule based on the historical service data.

Optionally, the second determining module includes:

the conversion unit is used for carrying out format conversion on the historical service data to obtain the historical service data in a key-value form; wherein, the key characterizes the field, the value characterizes the correspondent data value of the field;

The third determining unit is used for determining the data characteristics corresponding to the fields according to the data values corresponding to the fields;

and the determining unit is used for determining a historical data characteristic detection rule based on the data characteristics corresponding to the fields.

Optionally, the fourth determining unit is specifically configured to:

if the field is of an enumerated type, determining a data value corresponding to the field as a data characteristic corresponding to the field;

if the field is a field of a non-enumeration type, determining data characteristic description information corresponding to the field according to a data value corresponding to the field, and determining the data characteristic description information corresponding to the field as a data characteristic corresponding to the field.

Optionally, the determining unit is specifically configured to:

combining the data characteristics corresponding to each field in each piece of historical service data to obtain a data characteristic combination result;

and respectively determining each data characteristic combination result as a historical data characteristic detection rule.

The data change monitoring device in the embodiment of the present disclosure may further execute the method executed by the data change monitoring device in fig. 1 to 6, and implement the functions of the data change monitoring device in the embodiment shown in fig. 1 to 6, which are not described herein.

According to the monitoring device for data change provided by the embodiment of the specification, whether the service data characteristics of the first service node are changed is monitored by detecting whether the service data generated by the first service node when the service is processed accords with the predetermined historical data characteristic detection rule, so that whether the service mode of the first service node is adjusted can be judged, and when the service data characteristics of the first service node are monitored to be changed, a second service node is timely notified, wherein the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

Further, based on the methods shown in fig. 1 to 6, the embodiment of the present disclosure further provides a data change monitoring device, as shown in fig. 8.

The monitoring device for data changes may vary widely due to configuration or performance, may include one or more processors 801 and memory 802, and may have one or more stored applications or data stored in the memory 802. Wherein the memory 802 may be transient storage or persistent storage. The application programs stored in memory 802 may include one or more modules (not shown), each of which may include a series of computer-executable instruction information in a monitoring device for data changes. Still further, the processor 801 may be configured to communicate with the memory 802 and execute a series of computer executable instruction information in the memory 802 on a monitoring device for data changes. The monitoring device for data changes may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input/output interfaces 805, one or more keyboards 806, etc.

In a specific embodiment, the data change monitoring device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instruction information in the data change monitoring device, and the execution of the one or more programs by the one or more processors includes computer executable instruction information for:

if the detection result indicates that the service data does not accord with the historical data characteristic detection rule, sending prompt information to a second service node; the prompt information is used for indicating the change of the data characteristics of the service data generated by the first service node; the second service node is a downstream node of the first service node.

Optionally, when the computer executable instruction information is executed, if the detection result indicates that the service data does not conform to the historical data feature detection rule, the following steps may be further executed:

determining a data characteristic detection rule corresponding to service data;

adding the data characteristic detection rule into a historical data characteristic detection rule set; wherein each historical data characteristic detection rule is included in the historical data characteristic detection rule set.

Optionally, the computer executable instruction information, when executed, detects whether the business data meets a predetermined historical data feature detection rule, including:

determining data characteristics of data values corresponding to fields in service data;

and respectively matching the data characteristics corresponding to the service data with each historical data characteristic detection rule to determine whether the service data accords with the historical data characteristic detection rules.

Optionally, when the computer executable instruction information is executed, if the service data includes an enumerated type field and a non-enumerated type field;

determining a data feature detection rule corresponding to service data, including:

determining a data value corresponding to an enumeration type field in the service data, and determining data characteristic description information corresponding to a non-enumeration type field in the service data;

And combining the data value and the data characteristic description information to obtain a data characteristic detection rule corresponding to the service data.

Optionally, when the computer executable instruction information is executed, if the history service data includes an enumeration type field and a non-enumeration type field;

each historical data characteristic detection rule is formed by combining a historical data value corresponding to each field of the enumerated type and historical data characteristic description information corresponding to each field of the non-enumerated type.

Optionally, when the computer executable instruction information is executed, acquiring service data generated by the first service node when processing the service, including:

acquiring generated service data from a first service node when the first service node processes a service;

or, obtaining service data from a database; wherein, the database stores the business data generated by the first business node when processing business.

Optionally, before the computer executable instruction information, when executed, obtains service data generated by the first service node when processing the service, the method further includes:

acquiring historical service data corresponding to a first service node;

A historical data feature detection rule is determined based on the historical business data.

Optionally, the computer executable instruction information, when executed, determines a historical data feature detection rule based on the historical business data, comprising:

performing format conversion on the historical service data to obtain the historical service data in a key-value form; wherein, the key characterizes the field, the value characterizes the correspondent data value of the field;

determining the data characteristics corresponding to each field according to the data values corresponding to each field;

and determining a historical data characteristic detection rule based on the data characteristics corresponding to the fields.

Optionally, when the computer executable instruction information is executed, determining the data feature corresponding to each field according to the data value corresponding to each field, including:

Optionally, when the computer executable instruction information is executed, determining a historical data feature detection rule based on the data features corresponding to the fields, including:

According to the monitoring equipment for data change provided by the embodiment of the specification, whether the service data characteristics of the first service node are changed is monitored by detecting whether the service data generated by the first service node when the service is processed accords with the predetermined historical data characteristic detection rule, so that whether the service mode of the first service node is adjusted can be judged, and when the service data characteristics of the first service node are monitored to be changed, a second service node is timely notified, wherein the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

Further, based on the method shown in fig. 1 to 6, the embodiment of the present disclosure further provides a storage medium, which is used to store computer executable instruction information, and in a specific embodiment, the storage medium may be a U disc, an optical disc, a hard disk, etc., where the computer executable instruction information stored in the storage medium can implement the following flow when executed by a processor:

Optionally, when the computer executable instruction information stored in the storage medium is executed by the processor, if the detection result indicates that the service data does not conform to the historical data feature detection rule, the following steps may be further executed:

determining a data characteristic detection rule corresponding to service data;

Optionally, the computer executable instruction stored on the storage medium, when executed by the processor, detects whether the service data meets a predetermined historical data feature detection rule, including:

Optionally, when the computer executable instruction information stored in the storage medium is executed by the processor, if the service data includes an enumerated type field and a non-enumerated type field;

Optionally, when the computer executable instruction information stored in the storage medium is executed by the processor, if the history service data includes an enumeration type field and a non-enumeration type field;

Optionally, the computer executable instruction information stored in the storage medium, when executed by the processor, obtains service data generated by the first service node when processing the service, including:

Optionally, before the computer executable instruction information stored in the storage medium is executed by the processor to obtain service data generated by the first service node when processing the service, the method further includes:

acquiring historical service data corresponding to a first service node;

Optionally, the storage medium stores computer executable instruction information that, when executed by the processor, determines a historical data feature detection rule based on historical traffic data, comprising:

Optionally, the computer executable instruction information stored in the storage medium, when executed by the processor, determines, according to the data value corresponding to each field, the data feature corresponding to each field, including:

Optionally, the computer executable instruction stored on the storage medium, when executed by the processor, determines a historical data feature detection rule based on the data features corresponding to the fields, including:

When the computer executable instruction information stored in the storage medium provided in the embodiments of the present disclosure is executed by the processor, it is monitored whether the service data characteristics of the first service node change by detecting whether the service data generated by the first service node when processing the service conforms to a predetermined history data characteristic detection rule, so that it can be determined whether the service mode of the first service node is adjusted, and when it is monitored that the service data characteristics of the first service node change, the second service node is timely notified, where the second service node is a downstream node of the first service node. By the method provided by the embodiment of the specification, the monitoring of the service mode of the upstream service node is realized, so that the downstream service node can timely learn the change condition of the upstream service node, so that corresponding adjustment can be timely made, and the occurrence of service processing fault conditions is reduced.

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present application.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instruction information. These computer program instruction information may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instruction information, which is executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instruction information may also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instruction information stored in the computer readable memory produce an article of manufacture including instruction information means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instruction information may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instruction information which is executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instruction information, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The application may be described in the general context of computer-executable instruction information, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.

Claims

1. A method of monitoring data changes, the method comprising:

2. The method of claim 1, further comprising, if the detection result indicates that the business data does not meet the historical data feature detection rule:

determining a data characteristic detection rule corresponding to the service data;

adding the data feature detection rule to a historical data feature detection rule set; wherein the set of historical data feature detection rules includes each historical data feature detection rule.

3. The method of claim 1, the detecting whether the business data meets a predetermined historical data feature detection rule, comprising:

determining data characteristics of data values corresponding to fields in the service data;

4. The method of claim 2, if the traffic data includes an enumerable type field and a non-enumerable type field;

the determining the data characteristic detection rule corresponding to the service data comprises the following steps:

5. The method according to any of claims 1-4, wherein the acquiring service data generated by the first service node when processing the service comprises:

acquiring generated service data from the first service node when the first service node processes the service;

or acquiring the service data from a database; wherein, the database stores the business data generated by the first business node when processing business.

6. The method according to any of claims 1-4, the method further comprising, prior to the acquiring of traffic data generated by the first traffic node when processing traffic:

acquiring historical service data corresponding to the first service node;

and determining the historical data characteristic detection rule based on the historical service data.

7. The method of claim 6, the determining the historical data feature detection rule based on the historical business data comprising:

performing format conversion on the historical service data to obtain historical service data in a key-value form; wherein, the key characterizes the field, the value characterizes the data value that the said field corresponds to;

and determining the historical data characteristic detection rule based on the data characteristics corresponding to the fields.

8. The method of claim 7, wherein determining the data characteristic corresponding to each field according to the data value corresponding to each field comprises:

if the field is an enumerated field, determining a data value corresponding to the field as a data characteristic corresponding to the field;

9. The method of claim 7, wherein determining the historical data feature detection rule based on the data feature corresponding to each field comprises:

10. A device for monitoring changes in data, the device comprising:

11. The apparatus of claim 10, further comprising, if the detection result indicates that the service data does not meet the historical data feature detection rule:

the adding module is used for adding the data characteristic detection rule into a historical data characteristic detection rule set; wherein the set of historical data feature detection rules includes each historical data feature detection rule.

12. The apparatus of claim 10, the detection module comprising:

a first determining unit, configured to determine a data characteristic of a data value corresponding to each field in the service data;

13. The apparatus of claim 11, if the traffic data includes an enumerable type field and a non-enumerable type field;

the first determining module includes:

a second determining unit, configured to determine a data value corresponding to an enumerable type field in the service data, and determine data feature description information corresponding to a non-enumerable type field in the service data;

14. The apparatus of any of claims 10-13, further comprising:

And the second determining module is used for determining the historical data characteristic detection rule based on the historical service data.

15. A monitoring device for data changes, comprising:

a processor; and

16. A storage medium storing computer-executable instructions that when executed implement the following: